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Abstract 

Many  nonlinear  systems  can  be  interpreted  as  linear  transformations  between  vec¬ 
tor  spaces  under  appropriate  definitions  for  the  vector  operations  on  the  inputs  and 
outputs.  The  class  of  systems  which  can  be  represented  in  this  way,  is  discussed  here. 
This  class,  referred  to  as  the  class  of  homomorphic  systems,  is  shown  to  include  all 
invertible  systems.  Necessary  and  sufficient  conditions  on  a  noninvertible  system  such 
that  it  is  a  homomorphic  system,  are  derived. 

A  canonic  representation  of  homomorphic  systems  is  presented.  This  representa¬ 
tion  consists  of  a  cascade  of  three  systems,  the  first  and  last  of  which  are  determined 
only  by  the  vector  space  of  inputs  and  the  vector  space  of  outputs,  respectively.  The 
second  system  in  the  canonic  representation  is  a  linear  system.  Necessary  and  suf¬ 
ficient  conditions  are  presented  under  which  all  of  the  memory  in  the  system  can  be 
concentrated  in  the  linear  portion  of  the  canonic  representation.  A  means  for  classi¬ 
fying  homomorphic  systems,  suggested  by  the  canonic  representation,  is  discussed. 
This  means  of  classification  offers  the  advantage  that  systems  within  a  class  differ 
only  in  the  linear  portion  of  the  canonic  representation.  Applications  of  the  theory  are 
considered  for  a  class  of  nonlinear  feedback  systems. 
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I.  INTRODUCTION 


The  analysis  and  characterization  of  linear  systems  rely  heavily  on  the  principle  of 
superposition  which  these  systems  satisfy.  The  superposition  integral  and  system  func¬ 
tion  representations,  for  example,  are  a  direct  consequence  of  the  fact  that  the  response 
of  a  linear  system  to  each  input  in  a  set  of  inputs  determines  the  response  to  any  linear 
combination  of  inputs  in  that  set. 

In  contrast,  nonlinear  systems  do  not  satisfy  this  principle  of  superposition.  The 
determination  of  a  means  for  representing  nonlinear  systems  has  been  the  subject  of  con¬ 
siderable  attention  for  many  years.  In  1887,  Volterra*  formulated  a  functional  expansion 
of  continuous  nonlinear  operators  in  the  form  of  a  generalization  of  the  Taylor's  series 
expansion  of  a  nonlinear  function.  This  expansion,  applied  to  systems,  provides  a  repre¬ 
sentation  of  the  system  operator.  A  representation  of  time -invariant,  realizable  non- 
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linear  systems  was  presented  by  Wiener,  in  1958.  In  his  theory,  system  response  to 
shot  noise  is  used  to  determine  the  parameters  of  the  representation.  Wiener's  formu¬ 
lation  is  based  on  the  expansion  of  the  past  of  an  input  in  terms  of  Laguerre  functions , 
which  was  first  presented  by  Cameron  and  Martin.  The  response  of  a  realizable  time- 
invariant  nonlinear  system  is  expressed  as  a  nonlinear  combination  of  the  coefficients 
in  this  expansion.  Cameron  and  Martin  represented  the  nonlinear  combination  cf  these 
coefficients  in  terms  of  Hermite  functions.  An  alternative  representation  of  the  non¬ 
linear  combination  of  the  coefficients  was  developed  by  Bose/*  in  1956.  Much  of  the 
research  being  conducted,  at  present,  on  the  representation  of  nonlinear  systems  is 
based  on  Wiener's  original  ideas. 

This  report  presents  a  different  approach  to  the  characterization  of  nonlinear  sys  - 
terns,  which  is  based  on  linear  algebra.  The  fact  that  vector  spaces  of  time  functions 
can  be  constructed  under  a  variety  of  choices  for  the  definitions  of  vector  addition  and 
scalar  multiplication  permits  many  nonlinear  systems  to  be  representable  as  linear 
transformations  between  vector  spaces.  Specifically,  if  f  ^ (t)  and  f2(t)  represent  any 
two  system  inputs,  let  f^(t)  o  f^t)  denote  the  combination  of  these  inputs  under  a  spec¬ 
ified  rule,  such  as  addition,  multiplication  or  convolution.  If  c  is  any  scalar,  let  c^^t) 
denote  the  combination  of  the  scalar  c  and  the  input  fj(t).  Similarly,  let  □  denote  a 
rule  of  combination  between  any  two  outputs,  and  /  denote  a  rule  of  combination  between 
any  scalar  and  any  output.  If  the  system  inputs  constitute  a  vector  space  when  o  is  inter¬ 
preted  as  vector  addition  and  >  is  interpreted  as  scalar  multiplication,  and  if  the  system 
operator  «J>  has  the  property  that 

4> (f J (t)  O  f2(t)]  =  «|>[f J (t)]  D  *[f2<t)3 

and 

9[c>fiW]  =  c/^Wj, 

then  it  is  representable  as  a  linear  transformation  between  vector  spaces.  In  the  case 
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for  which  the  operations  o  and  o  are  addition  and  the  operations  >  and  /  are  multi¬ 
plication,  the  system  is  a  linear  system.  Otherwise,  the  system  is  in  general  nonlinear. 
Systems  representable  in  this  way  are  referred  to  here  as  homomorphic  systems,  a 
term  motivated  by  the  algebraic  definition  of  a  homomorphic  mapping  between  vector 
spaces. 

The  algebraic  fundamentals  for  the  study  of  homomorphic  systems  are  presented  in 
Section  II.  This  presentation  sets  down  the  basic  postulates  of  linear  algebra  and  devel¬ 
ops  the  algebraic  theorems  that  are  needed  in  the  study  of  homomorphic  systems. 

In  Sections  III  and  IV  the  properties  of  linear  transformations  are  used  to  investigate 
homomorphic  systems.  It  is  shown  in  Section  III  that  every  invertible  system  is  homo¬ 
morphic  for  any  choice  of  the  input  vector  space.  For  noninvertible  systems,  necessary 
and  sufficient  conditions  are  derived  under  which  the  systems  are  homomorphic. 

In  Section  IV  a  canonic  representation  for  homomorphic  systems  is  developed.  This 
representation  consists  of  a  cascade  of  three  systems,  the  first  and  last  of  which  are 
determined  entirely  by  the  input  and  output  vector  spaces,  respectively.  The  second 
system  in  the  representation  is  a  linear  system.  Necessary  and  sufficient  conditions 
are  derived  in  Section  IV  under  which  the  first  and  last  systems  in  the  canonic  repre¬ 
sentation  are  memoryless.  Section  IV  also  includes  a  discussion  of  the  use  of  the  can¬ 
onic  representation  in  treating  a  class  of  nonadditive  feedback  systems. 

The  canonic  representation  presented  here  offers  a  convenient  means  for  classifying 
homomorphic  systems.  Systems  within  a  given  class  differ  only  in  the  linear  portion  of 
their  canonic  representation  and  hence  a  comparison  of  systems  within  a  class  reduces 
to  a  comparison  of  linear  systems.  Alternatively,  the  analysis  of  a  homomorphic  system, 
when  its  class  is  known,  reduces  to  the  analysis  of  a  linear  system. 
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II.  VECTOR  SPACES  AND  LINEAR  TRANSFORMATIONS 


2.  1  INTRODUCTION 

The  results  presented  in  this  report  draw  heavily  on  the  notation,  concepts,  and 
theorems  of  linear  algebra.  Hence  it  is  appropriate  to  introduce  the  fundamental  prop¬ 
erties  of  vector  spaces  and  linear  transformations,  and  to  remind  the  reader  who  is 
familiar  with  vector  spaces  of  the  properties  used  here.  Although  proofs  of  all  theorems 
have  been  included,  it  is  the  theorems  themselves  that  will  assume  primary  importance. 
For  the  reader  who  is  familiar  with  linear  algebra,  a  quick  review  should  be  adequate 

to  convey  the  point  of  view  toward  linear  algebra  which  is  used  in  this  report.  Many  of 
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the  theorems  and  proofs  presented  here  have  been  published  by  others.  Some  of 
these  were  reformulated  in  terms  that  were  more  meaningful  within  the  context  of  this 
report.  The  remainder  of  the  theorems  were  formulated  and  proved  specifically  for  the 
purposes  of  this  report. 

2.  2  GROUPS,  FIELDS,  AND  VECTOR  SPACES 

It  is  convenient  to  introduce  the  postulates  of  the  algebra  of  vector  spaces  by  intro¬ 
ducing  the  postulates  of  groups,  which  constitute  a  weaker  algebraic  system. 

DEFINITION  1:  A  group  G  is  a  collection  of  objects  for  which  a  binary  operation 
*  is  defined,  subject  to  the  following  postulates: 

1.  If  a  and  b  are  in  G,  then  a*b  is  in  G. 

2.  If  a,  b,  and  c  are  in  G,  then 

{a*b)*c  =  a*(b*c). 

3.  There  exists  a  unique  element  e  in  G,  called  the  identity  element,  such  that  for 
all  a  in  G, 

a*e  =  e*a  =  a. 

4.  For  every  element  a  in  G,  there  exists  a  unique  element  a  *  in  G,  such  that 

-1  -1* 
a*a  =  a  *a  =  e. 

If  the  group  G  has  the  property  that  for  all  elements  a  and  b  in  G,  a*b  =  b*a,  then  G 
is  referred  to  as  a  commutative  or  Abelian  group.  Often  when  discussing  Abelian  groups, 
the  binary  operation  associated  with  the  group  is  denoted  +  to  emphasize  the  fact  that 
the  group  operation  is  commutative.  In  this  case,  the  identity  element  is  referred  to 
as  the  zero  element.  This  notation  will  be  used  now.  Little  confusion  results  when  the 
elements  of  the  groups  are  abstract  objects.  When  ihe  group  elements  are  real  or  com¬ 
plex  numbers  or  functions,  however,  the  reader  should  remember  that  the  element  a+b 
in  the  group  is  not  necessarily  the  addition  of  the  numbers  (or  functions)  a  and  b.  The 
algebraic  postulates  can  be  satisfied,  for  example,  if  a+b  is  defined  as  the  product  of 
the  numbers  or  functions  a  and  b. 
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DEFINITION  2:  A  field  F  is  a  collection  of  objects  associated  with  which  there  are 
two  binary  operations.  The  first  operation  is  denoted  a+b.  The  second  operation  is 
denoted  a.b.  These  operations  satisfy  the  following  postulates: 

1.  The  field  F  is  an  Abelian  group  under  the  operation  +. 

2.  The  nonzero  elements  of  F  are  an  Abelian  group  under  the  operation  •  ,  The 
identity  under  this  operation  is  denoted  by  1 . 

3.  For  any  a,  b,  and  c  in  F, 

a  *  (b+c)  =  a  •  b  +  a  *  c. 

It  should  again  be  mentioned  that  the  operations  +  and  ■  ,  although  satisfying  the  same 
algebraic  postulates  satisfied  by  addition  and  multiplication  of  numbers  or  functions, 
are  not  necessarily  restricted  to  be  these  operations  when  the  elements  of  the  field  are 
chosen  to  be  real  or  complex  numbers  or  functions. 

DEFINITION  3:  A  vector  space  consists  of 

1.  A  field  F  of  scalars; 

2.  A  set  V  of  objects  called  vectors  and  a  binary  operation  called  vector  addition 
(to  be  denoted  +)  under  which  the  set  V  constitutes  an  Abelian  group;  and 

3.  An  operation  called  scalar  multiplication  which  associates  with  each  scalar  c  in 
F  and  vector  v  in  V  a  vector  cv  in  V  called  the  product  of  c  and  v  in  such  a  way  that 

(a)  lv  =  v  for  every  v  in  V 

<b>  (Cj-c^v  =  Cj(c2vl 

(c)  c(Vj+V2)  =  CVj  +  CV2 

(d)  (Cj+C^V  »  CjV  +  c2v. 

The  inverse  of  a  vector  v  is  denoted  -v.  It  can  easily  be  verified  that 

(“T) v  =  -v 

and 

(0)v  =  0 

DEFINITION  4:  Let  V  be  a  vector  space  over  the  field  F.  A  subspace  of  V  is  a 
subset  Vj  of  V  which  is  itself  a  vector  space  over  F  with  the  operations  of  vector  add¬ 
ition  and  scalar  multiplication  on  V. 

A  simple  example  of  a  subspace  of  V  is  the  space  consisting  of  the  zero  vector 
alone.  When  verifying  that  a  subset  of  a  vector  space  is  a  subspace  we  need  not  verify 
all  of  the  postulates  of  a  vector  space,  since  many  of  the  postulates  are  implied  by  the 
fact  that  the  set  is  a  subset  of  a  vector  space.  In  general,  it  is  sufficient  to  show  that 
if  Vj  and  v2  are  in  Vj,  then  any  linear  combination  of  Vj  and  v2  is  also  in  Vj. 

DEFINITION  5:  A  vector  space  can  be  extended  to  what  is  termed  an  inner  product 
space  by  defining  the  inner  product  between  any  two  vectors  in  the  space.  The  inner 
product  assigns  to  each  ordered  pair  of  vectors  Vj,v2  hi  V  a  scalar  (vj»v2^  hi  F  in 
such  a  way  that 
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1.  (v1+v2>v3)  =  (v1,v3)  +  (v2,v3). 

2.  (CVjfV2)  =  C'(v1#  v2). 

3.  (vj*v2^  =  ^v2*vi^‘  w^ere  l3ar  denotes  complex  conjugation. 

4.  (v,  v)  >  0  if  v  *  0. 

Two  vectors  having  the  property  that  their  inner  product  is  zero  are  said  to  be  orthog¬ 
onal.  The  length  of  a  vector  v  in  an  inner  product  space  is  defined  as  the  square  root 
of  (v,v)  and  is  denoted  ||v||.  A  set  of  vectors  that  arc*  orthogonal  to  each  other  and  have 
unit  length  is  termed  an  orthonormal  set. 

Vector  spaces  can  be  constructed  for  which  the  elements  are  functions  of  time,  but 
for  which  the  operations  of  vector  addition  and  scalar  multiplication  do  not  correspond 
to  addition  of  time  functions  and  multiplication  of  time  functions  by  scalars.  For 
example,  consider  the  set  of  all  time  functions  of  the  form  v  =  e  ,  where  n  is  a  real 
number.  If  we  associate  with  this  set  the  field  of  real  numbers  and  define  vector  addi¬ 
tion  and  scalar  multiplication  as 

n.t  n?t 

Vj  +  v2  =  vl(t)  vz(t)  =  e  e 


and 


cv  =  [v(t)jc  =  [ent]C, 


then  this  set  constitutes  a  vector  space. 

Many  of  the  important  and  useful  theorems  concerning  vector  spaces  and  linear 

transformations  between  vector  spaces  stem  from  the  fact  that  a  set  of  basis  vectors 

can  always  be  found  for  a  finite-dimensional  vector  space.  The  notion  of  a  basis  is 

presented  in  the  following  definitions  and  theorems. 

DEFINITION  6:  Let  V  be-  a  vector  space  over  F.  A  finite  subset  S  of  V  is  said  to 

be  linearly  dependent  if  there  exist  scalars  c^,  c2,  . . .  ,  c^  in  F,  not  all  of  which  are  0, 

and  distinct  vectors  v. ,  v„,  . . , ,  v  in  S  such  that 

i  «  rx 


clvl  +  C2VH  +  ' '  ‘  +  cnvn  “  °‘ 


A  set  that  is  not  linearly  dependent  is  called  linearly  independent.  If  a  set  of  vectors  is 
infinite,  then  it  is  said  to  be  a  linearly  independent  set  if  every  finite  subset  of  it  is 
linearly  independent.  A  set  of  vectors  in  V  which  is  linearly  independent  and  spans  V, 
that  is,  is  such  that  any  vector  in  V  can  be  expressed  as  a  finite  linear  combination  of 
the  vectors  in  the  set,  is  referred  to  as  a  basis  for  V; 

It  can  be  verified  that  any  set  containing  the  zero  vector  is  a  linearly  dependent  set, 
and  that  the  representation  for  any  vector  in  terms  of  a  basis  is  unique. 

THEOREM  1:  Let  V  be  a  vector  space  for  which  the  finite  set  of  vectors  v^,  v2, 

. . . ,  vfi  forms  a  basis.  Then  any  other  basis  for  V  contains  exactly  n  vectors. 
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PROOF':  First,  we  shall  show  that  any  set  of  vectors  containing  more  than  n  ele¬ 
ments  must  be  linearly  dependent.  To  this  end,  let  w^,  w2,  . . .  ,  wm  be  a  set  of  m  vec¬ 
tors  from  V  wiih  m  >  n.  Since  the  set  v, ,  v_,  . , . ,  v  is  a  basis,  each  of  the  vectors 

1  2’  n 

is  expressible  as  a  linear  combination  of  the  vectors  vj,  v^,  . . .  ,  v^;  that  is. 


n 

w.  =  ^  c..v.. 

i  L  31  3 
3=1 


Let  Xj,  X2*  •  -  -  *  l36  scalars  from  F.  Then 


Since  m  >  n,  the  set  of  equations 


m 

Cj.X.  =  0,  j  =  1,  2 . n 

i=l 


has  a  nontrivial  solution  for  the  unknowns  X^,  . . . ,  Xm_  Thus  a  set  of  scalars  can 

be  found,  not  all  of  which  are  zero,  with  the  property  that 

m 

1  Vi  *  «• 

i=l 

Hence  the  set  w^,  w.,,  . . .  ,  wm  cannot  be  a  linearly  independent  set;  therefore,  no  lin¬ 
early  independent  set  in  V,  in  particular,  no  basis  for  V  can  contain  more  than  n  vec¬ 
tors.  Now  assume  that  there  exists  a  basis  with  p  vectors,  where  p  <  n.  Then,  by 
virtue  of  the  discussion  above,  there  could  not  exist  a  basis  with  more  than  p  vectors. 

But  the  set  of  basis  vectors  v . ,  v_,  . .  . ,  v  has  more  than  p  vectors.  Thus  there  could 

1  2  n  r 

not  have  existed  a  basis  with  p  vectors,  where  p  <  n. 

By  virtue  of  Theorem  1 ,  we  see  that  any  basis  for  a  vector  space  with  a  finite  num¬ 
ber  of  basis  vectors  contains  the  same  number  of  vectors.  It  is  this  property  of  vector 
spaces  that  permits  the  definition  of  the  dimension  of  a  vector  space  independently  of 
any  particular  basis. 

DEFINITION  7:  The  dimension  of  V  is  dei.ned  to  be  the  number  of  elements  in  a 
basis  of  V.  The  dimension  of  a  vector  space  V  will  be  denoted  dim  V. 

Many  of  the  theorems  of  linear  algebra  center  around  finite-dimensional  vector 
spaces.  Infinite-dimensional  vector  spaces  present  special  problems  and  many  theorems 
concerning  these  spaces  require  special  restrictions  on  the  spaces.  Infinite-dimensional 
vector  spaces  will  be  considered.  Before  doing  so,  it  is  important  that  the  properties 
of  finite-dimensional  vector  spaces  be  understood  so  that  the  extensions  of  these  prop¬ 
erties  to  infinite-dimensional  spaces  will  be  clearer. 
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THEOREM  2;  Let  S  be  a  linearly  independent  subset  of  a  finite-dimensional  vector 
space  V.  Then  S  is  part  of  a  basis  for  V. 

PROOF:  Let  VV  be  the  subspace  of  V  spanned  by  S.  Then,  by  definition,  S  is  a 
basis  for  W.  If  W  =  V,  then  S  is  also  a  basis  for  V  and  the  proof  is  complete.  If  W 
is  a  proper  subspace  of  V  {that  is,  contained  In  but  not  equal  to  V),  let  vj  be  a  vector 
that  is  in  V  but  not  in  W.  The  set  {v^.s}  obtained  by  adjoining  v^  to  S  is  linearly  inde¬ 
pendent,  for  if  Vj  could  be  expressed  as  a  linear  combination  of  vectors  in  S,  then  v^ 
would  be  in  W. 

Let  Vj  be  the  subspace  of  V  spanned  by  {v^.S}.  If  Vj  =  V,  then  {v^.S}  is  a  basis 
for  V,  since  it  is  a  basis  for  Vj.  If  Vj  is  a  proper  subspace  of  V,  let  v 2  be  a  vector 
that  is  in  V  but  is  not  in  Vj.  Then  the  set  {v^.v^.S}  must  be  linearly  independent.  Let 
this  process  be  continued  until  the  set  {vn,v  ^ .  .v^.v^  ,s}  contains  exactly  dim  V  ele¬ 
ments.  Since  these  vectors  are  all  linearly  independent,  the  space  spanned  by  this 

set  must  be  all  of  V;  otherwise,  a  vector  v  . ,  that  is  in  V  but  not  in  V  could  be 

n+ 1  n 

adjoined  to  this  set,  forming  a  linearly  independent  set  with  more  than  dim  V  elements. 
In  the  proof  of  Theorem  1,  however,  it  was  shown  that  no  linearly  independent  set  of  V 
could  contain  more  than  dim  V  elements. 

By  the  procedure  above,  then,  the  set  S  was  extended  to  a  basis  of  V  and  hence  is 
itself  part  of  a  basis. 

It  follows  from  Theorem  2  that  if  Vj  is  a  proper  subspace  of  a  finite-dimensional 
vector  space  V,  then  dim  Vj  <  dim  V.  This  follows  from  the  fact  that  any  basis  for  Vj 
is  part  of  a  basis  for  V.  Since  is  a  proper  subspace  of  V,  however,  a  set  of  vectors 
must  be  adjoined  to  the  set  of  basis  vectors  for  Vj  to  form  a  basis  for  V,  these  vectors 
being  chosen  by  the  procedure  given  in  the  proof  of  Theorem  2. 

The  existence  of  a  basis  leads  to  a  representation  of  a  vector  as  an  n-tuple  of 
s  calars.  This  representation  comes  about  by  expressing  every  vector  in  terms  of  an 
ordered  basis  for  the  space  V.  A  one-to-one  correspondence  can  then  be  drawn  between 
any  vector  in  V  and  the  n-tuple  of  the  coordinates  expressing  this  vector  in  terms  of  the 
ordered  basis.  This  is  done  formally  by  first  defining  a  vector  space,  the  elements  cf 
which  are  n-tuples  having  entries  from  the  field  F. 

DEFINITION  8:  The  vector  space  Fn  is  defined  to  be  the  vector  space  having  ele¬ 
ments  of  the  form  {aj,  a2> _ ,  a^},  where  a^ ,  a2,  ....  a^  are  scalars  from  the  field  F. 

The  vector  addition  of  two  vectors  {a. ,  a_,  —  ,  a  )  and  (b, ,  b„, _ _  b  )  in  Fn  is  defined  as 

*  c  n  id  n 

*al  ’  a2*  *  *  ‘  *  an*  *  *bl  *  b2«  •  *  -  •  bn)  =  (ai+bl  ’  aZ+b2 . an+bn*' 

The  scalar  multiplication  of  a  vector  {aj,a2» ...» an)  in  F11  with  a  scalar  c  in  F  is 
defined  as 

c{a1#a2, . . .  ,an)  =  (ca^ca^. . .  ,  can>. 

Let  V  be  a  finite  dimensional  vector  space  over  F  with 
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dim  V  =  n. 


If  Vj.  v2,  . . . ,  v  is  an  ordered  basis  for  V,  then  any  vector  v  can  be  expressed  in  the 
form 


v  =  &jVj  +  a2v2  + 


+  a  v  . 
n  n 


If  we  draw  a  correspondence  between  v  and  the  n-tuple  (aj,a2>. .  .,an)  in  Fn,  then  this 
correspondence  will  be  one-to-one,  since  any  vector  v  in  V  can  be  expressed  in  one 
and  only  one  way  in  terms  of  a  given  ordered  basis. 


2.3  QUOTIENT  SPACES 


A  concept  that  will  have  application  in  the  discussion  of  linear  transformations,  and 

also  in  the  discussion  of  homomorphic  systems,  is  that  of  a  quotient  space.  The  essence 

of  the  idea  is  to  divide  a  vector  space  V  into  sets  of  vectors,  S  ,  S_,  . . . ,  S  with  the 

i  b  n 

property  that  the  sets  Sj,  S2 . S^  can  themselves  be  considered  as  vectors  in  a 

vector  space. 

DEFINITION  9:  Let  Vj  be  a  subspace  of  V,  and  Vj  be  a  vector  in  V.  Let  Sj  be 
the  set  of  all  vectors  in  V  which  can  be  expressed  in  the  form  Vj  +  w  for  some  w  in 
Vj.  Denote  this  set  by  Vj  +  Vj.  Tills  set  is  called  a  coset  of  V  generated  by  Vj  and 
Vj.  The  set  of  all  cosets  generated  by  vectors  v  in  V,  and  the  subspace  Vj  is  called 
a  quotient  space  of  V,  modulo  Vj,  and  is  denoted  by  V/Vj. 

THEOREM  3;  Let  V  be  a  vector  space  over  F,  and  Vj  be  a  subspace  of  V.  The 
quotient  space  V/Vj  is  a  vector  space  over  F  with  the  following  definitions  of  vector 
addition  and  scalar  multiplication: 

(Vj+V.)  +  (Vj+Vj)  a  [vj+v2]  +  Vj 

CiVj+Vj)  a  [cvjl  +  Vj. 

PROOF:  To  show  that  V/Vj  is  a  vector  space  we  must  verify  that  all  the  postulates 
of  a  vector  space  as  presented  in  Definition  3  are  satisfied. 

1.  The  set  V/Vj  is  an  Abelian  group,  since 

(a)  If  Vj  +  Vj  and  v2  +  Vj  are  any  two  cosets  of  V,  then  their  vector  sum 
<v  +v2>  +  Vj  is  also  a  coset  of  V,  and 

(Vj+Vj)  +  (v2+Vj)  =  (Vj+v2)  +  Vj 

=  (V2+Vj)  +  Vj 

=  (v2+Vj)  +  (Vj+Vj). 

(b)  Let  Vj  +  Vj,  v2  +  Vj,  and  v3  +  Vj  be  cosets  of  V;  then 
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[(v,+V.)+{v^+Vj]  +  (v«+V. 5  =  fiv.+v.i+V.l  +  fv  +V  } 

1  i  C.  I  -  3  1  *•12’  1  J  1 

=  [(Vjfv2)+V3]  +  Vj 

=  ^v1+«v2+v3n  +  V! 

=  (Vj+Vj)  +  [(Vj+VjJ+^j+Vj)!. 

(c)  It  can  be  verified  that  the  coset  (O+Vj)  has  the  properties  required  for  an 
identity  element  in  V/Vj. 

(d)  Let  v  +  Vj  be  a  coset  of  V;  then  (-v)  +  Vj  is  the  inverse  of  v  +  Vj, 

2.  The  four  properties  required  of  scalar  multiplication  are  satisfied. 

(a)  lfv+Vj)  =  lv  +  Vj  =  v-+  Vj. 

(b)  (Cj-c^tv+Vj)  =  Cj  ■  c2v  +  Vj  =  c1[c2(v+V1)]. 

<c)  c[(Vj+V  j)+(v2+V  j )]  =  ctlVj+v^+Vj] 

=  [(cVj+cv^+Vj] 

=  (cVj+Vj)  +  (cv2+Vj). 

(d)  (Cj+^Hv+Vj)  =  [(tj+c^v+Vj] 

=  [CjV+c^+VjJ 
=  (CjV+Vj)  +  {c2v+Vj). 

To  aid  in  understanding  the  concept  of  a  quotient  space,  consider  a  geometrical  two- 
dimensional  coordinate  system  in  the  x-y  plane.  All  vectors  in  this  plane  form  a  vector 
space,  which  we  can  denote  by  V.  Let  Vj  be  the  subspace  of  V  consisting  of  all  vec¬ 
tors  lying  in  the  x  direction.  Let  v  be  a  vector  in  V  having  a  component  in  the  y 
direction  equal  to  v  .  Then  the  coset  v  +  Vj  is  the  set  of  all  vectors  in  the  plane  having 
Vy  as  their  y  component.  The  quotient  space  V/Vj  is  the  set  of  all  such  cosets  for 

different  values  of  v  . 

y 

THEOREM  4:  Let  v  +  V,  and  v,  +  V,  be  any  two  cosets  in  V/V, .  If  these  cosets 

a  i  b  i  i 

have  any  element  in  common,  then  they  are  identical. 

PROOF:  Let  v  be  any  vector  that  is  in  v&  +  Vj  and  also  in  +  Vj.  Then  there 
exists  Vj  in  Vj  such  that 


v  = 


v  + 
a 


F 


and  there  exists  v’j  in  Vj  such  that 


v  =  v  +  V* 

b  1* 
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Hence, 


va=  vb+ 

Consequently, 

va  +  vi  =  {Vvi-vi)  +  vr 

But  -  Vj  is  in  V^;  hence. 


v  +  V.  =  v.  -r  V 
a  1  b  1 


2.4  LINEAR  TRANSFORMATIONS 

The  properties  of  the  class  of  nonlinear  systems  to  be  discussed  in  Sections  III  and 

IV  are  based  on  the  properties  of  linear  transformations  between  vector  spaces.  The 
theorems  to  be  derived  concerning  these  systems  will  be  for  the  most  part  an  interpre¬ 
tation  of  theorems  presented  here,  placed  in  the  context  of  the  discussion  of  Sections  III 
and  IV. 

DEFINITION  10:  Let  V  and  W  be  vector  spaces  over  the  field  F.  A  linear  trans¬ 
formation  from  V  into  VV  is  a  function  T  from  V  into  W  which  maps  every  vector  in 

V  onto  a  unique  vector  in  W  in  such  a  way  that 

TtcVj+v^  =  cTfVj)  +  T<v2) 

for  all  vectors  v^  and  v^  in  V  and  all  scalars  c  in  F.  A  linear  transformation  is  said 
to  be  one-to-one  and  onto  if  for  each  vector  w  in  W  there  is  one  and  only  one  vector  v 
in  V  such  that 

T(v)  =  w. 

A  linear  transformation  that  is  one-to-one  and  onto  is  invertible. 

If  a  transformation  is  onto  but  is  not  necessarily  one-to-one,  it  is  sometimes 
referred  to  as  a  homomorphic  transformation.  A  transformation  that  is  one-to-one  and 
onto  is  sometimes  referred  to  as  an  isomorphic  transformation,  that  is,  an  isomorphic 
transformation  is  an  invertible  homomorphic  transformation. 

DEFINITION  11:  The  domain  of  a  linear  transformation  is  the  set  of  vectors  V  over 
which  the  transformation  T  is  defined.  The  range  R  of  T  is  defined  as  the  set  of  vec¬ 
tors  in  W  onto  which  vectors  in  V  are  mapped,  that  is,  T[V]  =  R.  The  null  space  N  of 
T  is  the  set  of  vectors  in  V  that  map  onto  the  zero  vector  in  W,  that  is,  T[N]  =  0. 

An  example  of  a  linear  transformation  is  the  identity  mapping  I  of  V  onto  V  so  that 

I(v)  =  v,  for  every  v  in  V. 

The  domain  and  range  of  I  are  V.  The  null  space  of  I  is  the  zero  vector.  Another 
simple  example  of  a  linear  transformation  is  the  zero  transformation  Z  from  V  to  VV 
defined  as  Z(v)  =  0;  all  v  in  V.  The  domain  and  null  space  are  both  the  space  V.  The 
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range  of  £>  is  the  set  containing  only  the  zero  vector  in  W. 

THEOREM  5:  If  T  is  a  linear  transformation  from  V  to  W,  then  T(0)  =  0. 

PROOF:  T{0)  =  T{0+0)  =  T(0)  +  T{0).  Hence,  T{0)  =  0, 

THEOREM  6:  Let  T  be  a  linear  transformation  from  the  vector  space  V  into  the 
vector  space  W.  Then  the  range  of  T  is  a  subspace  of  W,  and  the  null  space  of  T  is 
a  subspace  of  V. 

PROOF:  Let  R  denote  the  range  of  T.  If  the  range  contains  only  one  vector,  then 
this  vector  must  be  the  zero  vector,  since  by  Theorem  5  the  zero  vector  must  be  in  the 
range.  But  the  zero  vector  alone  is  a  subspace  of  W,  and  hence  in  this  case  R  is  a 
subspace  of  W.  If  R  contains  more  than  one  vector,  then  we  must  show  that,  for  every 
pair  of  vectors  w^  and  w^  in  R,  the  vector  c^w^  +  ^  ^  *or  any  cj  ana  cz‘ 

But  if  Wj  is  in  R  and  w2  is  in  R,  then  there  are  {not  necessarily  unique)  vectors  Vj 
and  v2  in  V  such  that 

T{Vj)  =  w  j 

T(v2)  =  w2. 

The  vector  e^v^  +  c.,v2  is  in  the  domain  V  for  any  Cj  and  c2,  since  V  is  a  vector 
space.  Hence  Tfc^Vj+c^v^  is  in  R,  that  is,  CjWj  +  c2w2  is  in  R.  Consequently,  R 
is  a  vector  space. 

To  show  that  the  null  space  N  is  a  vector  space,  we  must  show  either  that  N  con¬ 
sists  of  the  zero  vector  alone  or  c,  Vj  +  C2V2  is  in  N  for  every  Cj  and  c2,  if  Vj  and 
v2  are  in  N. 

From  Theorem  5,  the  zero  vector  must  be  in  N.  Hence  if  N  contains  only  one  vec¬ 
tor,  it  must  be  the  zero  vector  and  consequently  N  is  a  subspace  of  V.  If  N  contains 
more  than  one  vector,  then  let  v^  and  v2  be  in  N,  that  is,  T(Vj)  =  0  and  T(v2)  =  0.  Then 

T(civi+c2»21  =  CjT(Vj)  +  c2T(v2) 

*  cl°  +  c2° 

=  0 


and  hence  CjVj  +  c2v2  is  in  N  for  any  Cj  and  c2. 

THEOREM  7:  Let  V  be  a  finite-dimensional  vector  space  over  F,  and  {vjtV2, . .  *»vn) 
be  a  basis  for  V.  Let  W  be  a  vector  space  over  F,  and  {wj,w2» . .  .»wn)  be  any  vectors 
in  W.  Then  there  is  one  and  only  one  linear  transformation  from  V  into  W  such  that 

T(v.)  =  wif  i  =  1 ,  2,  . . . ,  n. 

PROOF:  Any  vector  v  in  V  can  be  expressed  as 
n 
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c.w.,  then 
J  J 


where  the  coefficients  c.  are  uniaue.  If  we  define  Tlv)  =  ) 

J  *  U 

j=l 


T(v.)  =  w., 
1  1 


1,  2, 


n. 


It  can  be  verified  that,  with  this  definition,  T  is  a  linear  transformation.  To  prove 
uniqueness,  let  be  any  linear  transformation  with  the  property  that 

Tjlv.)  =  w.,  i  =  1,  2,  . . n. 


Then 


/n  \  n  n 

Ti|v)  =  Ti(  1  Ti) =  I  cjTi(vi1  s  1  cjwi‘ 

\ 3=1  /  3=1  3=1 


since  Tj  is  linear.  Hence  Tj(v)  =  T(v)  for  all  v;  consequently,  the  transformation  T 
defined  above,  is  unique. 

THEOREM  8:  Let  V  and  YV  be  vector  spaces  over  F,  end  T  be  a  linear  trans¬ 
formation  from  V  into  YV.  If  T  is  one-to-one  and  onto,  then  the  inverse  transformation 
T  *  is  a  linear  transformation  from  YV  onto  V. 

PROOF:  YVe  must  show  that  if  T  *{w^)  =  v^  and  T~*  (w^)  =  v^,  then  T~*  (cw^+w^)  = 
ct'i  +  v2.  The  fact  that  Tfcv^+v^  =  cw1  +  w2  follows  from  the  linearity  of  T.  Further¬ 
more,  cVj  +  v2  is  the  only  vector  in  V  that  maps  onto  cw^  +  w2>  since  T  is  one-to- 

one,  Hence  T-1  is  linear. 

DEFINITION  12:  A  linear  transformation  T  is  defined  to  be  nonsingular  if  the  null 
space  of  T  consists  of  the  zero  vector  alone. 

THEOREM  9:  Let  T  be  a  linear  transformation  from  V  into  YV.  Then  T  is  non¬ 
singular  if  and  only  if  T  carries  each  linearly  independent  subset  of  V  into  a  linearly 
independent  subset  of  YV. 

PROOF:  First,  suppose  T  is  nonsingular.  Let  the  set  _ .v^}  be  a  linearly 

independent  subset  of  V.  If 


=1T(V1>  +  C2T{V2}  +  •  •  *  +  CnT(vn)  =  °’ 


then 


T(C1V1+C2V2+-  •  -Vn1  = 

But  T  is  nonsingular  and  hence 

°lvl  +  C2V2  +  ■ "  +  Vn  =  °- 


Since  the  set  {vj,y2». .  .»vn)  is  linearly  independent,  this  then  requires  that  c.  =  0,  i  =  1, 
2,  — ,  n.  Hence  the  set  (T{Vj),T(v2),  . ,  .,T(vn)}  is  a  linearly  independent  subset  of  W. 
Next,  suppose  that  T  maps  every  linearly  independent  subset  of  V  onto  a  linearly 
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independent  subset  of  W.  Let  v  be  any  nonzero  vector  in  v.  Then  the  set  consisting 
only  of  the  vector  v  is  a  linearly  independent  subset  of  V.  Thus  the  set  in  W  consisting 
only  of  the  vector  T(v)  must  be  a  linearly  independent  subset  of  W  and  hence  must  be 
nonzero.  Hence,  T  is  nonsingular. 

THEOREM  10:  Let  V  and  W  be  finite-dimensional  vector  spaces  over  F,  with 
dim  V  =  dim  W  =  n.  If  T  is  a  liner  r  transformation  from  V  into  W,  then  the  following 
are  equivalent: 

(i)  T  is  invertible. 

(ii)  T  is  nonsingular. 

(iii)  If  (vi »v2‘  *  *  ',vn)  is  basis  for  V,  then  {T(Vj),T(v2) , . .  .,T{vn>}  is  a  basis 
for  W. 

PROOF:  It  should  be  clear  that  (i)  implies  (ii),  since  the  existence  of  an  inverse 
requires  that  the  transformation  be  one-to-one. 

(ii)  implies  (iii);  for  example,  assume  that  T  is  nonsingular.  If  (vj »v2» _ »vn) 

a  basis  for  V,  then  the  set  {t(Vj),T(v2) , . .  ,,T(vn)}  is  a  linearly  independent  subset  of 
VV.  Hence  it  is  part  of  a  basis  for  W.  But  any  basis  of  W  must  contain  exactly  n  vec¬ 
tors.  Hence  the  set  {T(Vj),T(v2),  . .  .,T(vn)}  must  be  a  basis  for  W. 

To  show  that  (iii)  implies  (i),  we  must  show  that  for  every  vector  w  in  W,  there  is 
a  unique  vector  v  in  V  such  that  T(v)  =  w.  Since  {T(Vj),T(v,), . .  .,T(vn)}  is  a  basis  for 
W,  the  vector  w  can  be  expressed  as 

n 

w  =  £  CjTOTj). 

3=1 


n 


Hence  the  vector  v  =  7  c.v.  is  such  that  T(v)  =  w.  Assume  that  there  is  some  other  vec- 

L  J  ) 

3=1 

tor  v  in  V  such  that  T(v  )  =  w.  If  v  is  expressed  in  terras  of  the  basis  {v  ,v_, . .  ,,v  }  as 
3  3  3,  i  4  n 


n 

=  >  a.v„ 
L  3  3 


3=1 


then 


n 

T,va>  =  w  =  I  ajTlvj'- 

j1  1 

But  the  set  {T(Vj),T(v2), _ .Tfv^)}  is  a  basis  for  W;  hence,  any  vector  in  W  can  be 

expressed  in  one  and  only  one  way  as  a  linear  combination  of  vectors  that  form  a 
basis.  Hence, 
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aj  "  cj*  3=  l,  c,  ....  n; 

consequently,  v  =  v. 

Theorems  7  and  10  together  imply  that  if  we  have  two  finite-dimensional  vector 
spaces  V  and  W  of  the  same  dimension,  then  an  invertible  linear  transformation  can 
always  be  defined  with  domain  V  and  range  W,  by  mapping  each  vector  in  a  set  of  basis 
vectors  for  V,  onto  a  different  basis  vector  in  W.  When  infinite-dimensional  vector 
spaces  are  discussed,  this  result  will  be  extended  to  state  that  an  invertible  linear  trans¬ 
formation  between  V  and  W  can  always  be  defined  when  a  one-to-one  correspondence 
can  be  drawn  between  basis  vectors  of  V  and  basis  vectors  of  W,  that  is,  that  two  vec¬ 
tor  spaces  V  and  W  are  isomorphic  whenever  a  one-to-one  correspondence  can  be  drawn 
between  basis  vectors  of  V  and  basis  vectors  of  W. 

THEOREM  IT.  Every  n-dimensional  vector  space  V  over  the  field  F  is  isomorphic 
to  the  sp  ce  Fn. 

PROOF:  Let  {v^.v^,. .  -*vn)  be  a  basis  for  V.  Then  every  vector  v  in  V  is  express¬ 
ible  in  the  form 

n 

v  *  I  Vk- 

i=l 

Let  T,  a  transformation  from  V  to  Fn,  be  defined  as 
T(v)  =  <clfc2, . .  .,cn) 

It  can  be  verified  that  T  is  linear,  one-to-one,  and  maps  V  onto  Fn. 

THEOREM  12:  Let  T  be  a  linear  transformation  from  V  onto  W  with  null  space 
N.  Then  the  quotient  space  V/N  is  isomorphic  with  W. 

PROOF:  Let  v  +  N  be  a  coset  of  V.  Define  the  mapping  T  from  V/N  to  W  as 

T(v+N)  =  T(v). 

We  must  first  show  that  this  mapping  is  well  defined,  that  is,  that  T[{v+n)+N}  =  T(v+N) 
for  any  v  in  V  and  any  n  in  N.  But 

T[{v+n)+N]  =  T(v)  +  T(n)  =  T(v)  =  T[v+N], 

since  n  is  in  the  null  space  of  T.  Hence,  T  is  well  defined.  T  is  linear,  since 
T[c  (v  j  +N) + (v2+N)  ]  =  TltcVj+v^+N] 

=  cT(Vj)  +  T(v2) 

=  cT[Vj+N]  +  T{v2+N}. 

The  null  space  of  T  is  the  coset  (0+N);  for  example,  assume  that  T(v+N)  =  T{v)  =  0. 
Then  v  is  in  N  and  hence  is  in  the  coset  (0+N).  Thus  T  is  nonsingular  and,  by  Theorem 
10,  is  then  invertible. 
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2.5  INFINITE-DIMENSIONAL  VECTOR  SPACES 


Many  of  the  theorems  that  have  been  presented  required  that  the  vector  spaces  under 
consideration  be  finite- dimensional.  In  Sections  III  and  IV  the  primary  concern  will  be 
with  infinite-dimensional  vector  spaces  and  linear  mappings  between  such  spaces.  In 
this  section,  therefore,  infinite-dimensional  vector  spaces  will  be  considered  and  some 
of  the  important  theorems  previously  discussed  for  finite-dimensional  spaces  will  be 
reformulated  for  the  infinite-dimensional  case. 

Many  of  the  properties  of  finite-dimensional  vector  spaces  relied  on  the  fact 
that  such  spaces,  by  definition,  always  possess  a  countable  basis.  Theorems  such 
as  Theorem  7,  for  example,  rely  heavily  on  the  fact  that  the  basis  is  countable. 
Although  it  can  be  shown  in  general  that  any  vector  space  possesses  a  basis, 
some  of  the  properties  to  be  discussed  in  the  rest  of  the  report  will  require 
that  the  vector  spaces  under  consideration  have  countable  bases.  Hence,  the  discus¬ 
sion  of  infinite-dimensional  vector  spaces  to  be  presented  here  will  be  restricted 
to  spaces  of  countably  infinite  dimensions. 

Even  with  the  restriction  that  the  bases  are  countable,  it  is  important  to  reconsider 
the  meaning  of  a  linear  combination  of  an  infinite  set  of  vectors.  We  wish  eventually  to 
carry  over  to  the  infinite-dimensional  case  a  similar  interpretation  of  the  meaning  of  a 
basis  as  a  set  of  vectors  spanning  the  space  and  providing  a  unique  representation  for 
any  vector  in  the  space. 

Many  of  the  problems  concerned  with  the  extensions  to  the  infinite-dimensional  case 
center  around  a  precise  definition  of  the  equality  of  two  vectors.  When  an  inner  product 
is  defined  on  the  space,  two  vectors  can  be  defined  to  be  equal  when  their  difference  is 
a  vector  of  length  zero.  This  definition  enables  many  of  the  desired  extensions  to  follow 
in  a  straightforward  manner.  An  infinite-dimensional  inner-product  space  with  certain 
convergence  properties  is  a  Hilbert  space.  These  spaces  have  been  extensively  studied 
and  are  the  most  straightforward  infinite-dimensional  extensions  of  finite- dimensional 
vector  spaces. 


DEFINITION  13:  Let  H  denote  an  inner-product  space.  H  is  defined  to  be  a 
Hilbert  space  if  for  any  sequence  of  vectors  in  H,  Vj ,  v^, . . . ,  v  , . . .  having  the 
property  that  Ivm”vn|J  —  0  as  m,n— <»,  there  exists  a  vector  v  in  H  such  that 
||  v  -v  ||  —  0  as  n  —  re. 


DEFINITION  14:  A  sequence  of  vectors  vfi  is  called  an  orthonormal  basis  for  a 
Hilbert  space  if  it  is  orthonormal  and  has  the  property  that  the  only  vector  in  H  which 
is  orthogonal  to  all  vectors  in  the  sequence  is  the  zero  vector.  A  Hilbert  space  with  an 
orthonormal  basis  is  called  a  separable  Hilbert  space. 

THEOREM  13:  If  vn  is  an  orthonormal  sequence  of  vectors  in  a  Hilbert  space  and 

co  n 


n 


is  a  sequence 


of  scalars  such  that  ^  |\jJ2  <  °°*  then  the  sequence  wr  =  ^  X 


kvk 


1 


1 
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w 

converges  to  a  limit  v  in  H  denoted  v  =  ^  K^v^. 


PROOF:  We  must  first  show  that  Ihe  sequence  wn  has  the  property  that  i!wm-wn!!  "*  ® 
as  m,n  —  oo.  Letting  m  =  n  +  p  for  some  integer  p,  we  have 


w  .  —  w 
1  n+p  n 


n+P 


I 


k  k 


n+l 


Since  the  sequence  vfi  is  an  orthonormal  sequence,  however, 


n+p 

y 

/ 

Uj 

n+l 


Vk 


n+p 

-2  m2- 


n+l 


But 


n+p 

I 

n+l 


n+p 

I 

1 


n+l 


I  M*-I  M2-£  W2- 


00 

Since  ^  lyP  <  ou*  however,  we  conclude  that 


I  I 

n+l 


XjJ  —  0  as  n  —  oo. 


Consequently,  lwm~wnii  —  0  as  m,n  —  <»;  hence,  there  exists  a  vector  v  in  H  such  that 


jjwn-v|j  —  0  as  n  —  oo,  that  is,  the  sequence  w^  converges  to  the  limit  vector  v. 

THEOREM  14:  Let  v  be  an  orthogonal  sequence  of  vectors  in  a  Hilbert  space. 

n  00 

Given  any  vector  v  in  H,  the  scalars  \k  =  (v,vk)  satisfy  the  inequality  ^  |xj2  <  °o. 

1 

PROOF:  Let  Vj, , . . ,  y.  be  orthogonal  vectors  in  H.  By  direct  expansion,  it  can 
be  verified  that  for  any  v  in  K, 


Since 


3 

V-Z  (v-vk)vk 
1 


"2  <>'-vk|vk 


=  H2  -  ^  Mv*vk)P- 


must  always  be  greater  than  or  equal  to  zero,  it  follows  that 


16 


INI2  *  X  l|v’vk|l2- 

1 

Since  this  is  true  for  any  j,  we  have 
00  00 
Z  i,v’vk>i2=  I  i\i! 


<  oo. 


THEOREM  15:  If  vn  is  an  orthonormal  basis  for  a  Hilbert  space  H,  then  each  vec¬ 


tor  v  can  be  expressed  as 

CO 

v=  2  !».' 


'Vk)Vk' 


PROOF:  We  note,  first,  that  the  infinite  sum  is  defined  in  Theorem  13,  since  the 

sequence  of  scalars  X  =  (v,v  )  was  shown  in  Theorem  14  to  have  the  required  prop- 

00 

erty  of  being  square  summable.  Consider  the  vector  w  =  v  -  {v.v^Jv^.  We  wish 

1 

to  show  that  this  vector  is  the  zero  vector.  But  (w,v.)  =  0  for  any  j,  since 

3 

(W,v.)  -  (vfv  )  -  (v,v.)  =  0; 
j  j  j 


consequently,  the  vector 


r  y 

!v->  (V, 

L  Y 


vk)vk 


is  orthogonal  to  every  vector  in  the  sequence 


v  .  From  Definition  14,  however,  the  only  vector  orthogonal  to  each  vector  in  an 
orthonormal  basis  is  the  zero  vector.  Hence 


00 


-Z- 


vk|vk  ■  ° 


or 


00 

v'  Z  (v*vk>vk- 
1 

These  theorems  provide  a  framework  for  the  infinite-dimensional  Hilbert  spaces 
that  are  to  be  considered.  Theorem  15  requires  the  existence  of  an  orthonormal  basis 
for  the  spaces.  This  set  of  basis  vectors  spans  H  by  virtue  of  Theorem  15.  It  can  be 
verified  that  this  set  of  vectors  is  linearly  independent,  by  virtue  of  the  property  that 
the  only  vector  orthogonal  to  the  entire  set  is  the  zero  yector.  In  general,  it  is  not  true 
that  every  Hilbert  space  possesses  an  orthonormal  basis. 

i-n  example  of  a  Hilbert  space  that  has  an  orthonormal  basis  is  the  space  of  square 
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integrabie  functions  cither  on  the  infinite  interval  {-«k+co)  nr  on  the  semi- infinite  inter- 

val  (0,+oo).  This  space  on  the  infinite  interval  is  usually  denoted  L  {— «>,  +oo) ,  and  on  the 

Z  2 

semi-xnfinite  interval  L  (0,+<»).  An  orthonormal  basis  for  L  (-oo,+oo)  is  the  set  of 
Hermite  functions.  An  orthonormal  basis  for  L  (0,+«>)  is  the  set  of  Laguerre  functions. 
In  both  of  these  spaces,  the  inner  product  is  defined  as  the  integral  over  the  interval  of 
the  product  of  the  functions. 

In  the  previous  discussion  on  finite-dimensional  vector  spaces,  it  was  shown  that  the 
domain  of  any  linear  transformation  is  a  vector  space.  If  the  domain  of  the  transfor¬ 
mation  is  a  Hilbert  space,  it  is  not  true  in  general,  however,  that  the  range  will  also  be 
a  Hilbert  space.  If  continuity  as  defined  below  is  imposed  on  the  transformation,  then, 
as  will  be  seen  in  the  following  discussion,  the  range  of  the  transformation  will  be  a 
separable  Hilbert  space  if  the  domain  is  a  separable  Hilbert  space. 

DEFINITION  15:  Let  V  and  W  be  inner  product  spaces,  and  '  be  a  linear  trans¬ 
formation  from  V  onto  W.  T  is  said  to  be  continuous  at  v  if  T(vn)  —  T(v)  as  n-«  if 

v  —  v  as  n  —  oo  where  v  is  a  sequence  of  vectors  in  V.  T  is  a  continuous  transfor- 

n  n  -  - 

mation  if  it  is  continuous  at  every  v  in  V.  Continuity  of  the  transformation  T  is  not  a 
severe  restriction  when  T  is  linear,  for  it  can  be  shown  that  T  will  always  be  contin¬ 
uous  if  it  is  bounded,  that  is,  if  there  exists  a  constant  M  >  0  such  that  for  every  vec¬ 
tor  v  in  H,  ||  X(v)  |j  ?£  M ||  v  j| .  (See,  for  example,  Berberian.8)  The  set  of  theorems 
that  follows  is  chosen  to  show  that  the  range  of  a  continuous  linear  transformation  is  a 
separable  Hilbert  space  if  the  domain  is  a  separable  Hilbert  space.  In  carrying  out  the 
proof  the  following  preliminary  results  will  be  demonstrated. 

(1)  The  null  space  N  of  T  is  a  closed  subspace  of  H,  that  is,  the  limit  of  any 

sequence  of  vectors  in  N  is  in  N  when  this  limit  exists. 

(2)  From  (1),  it  follows  that  H  can  be  decomposed  into  two  spaces  M  and  N.  The 
space  N  is  the  null  space  of  T,  and  the  space  M  is  the  space  of  all  vectors  in  H  which 
are  orthogonal  to  every  vector  in  N.  Any  vector  in  H  can  then  be  expressed  as  a  unique 
linear  combination  of  a  vector  in  M  and  a  vector  in  N. 

(3)  On  the  basis  of  (2) ,  it  will  be  shown  that  the  quotient  space  H/N  is  a  separable 
Hilbert  space,  and  that  the  linear  transformation  T  from  H/N  to  the  range  of  T  is  a 
continuous  transformation. 

(4)  It  follows  directly  from  (i)  that  the  range  of  T  is  a  separable  Hilbert  space. 
Steps  (l)-{4)  will  be  stated  forma  *  <  y  in  Theorems  16-19.  In  each  of  these  theorems  H 
denotes  a  separable  Hilbert  space,  and  T  denotes  a  continuous  linear  transformation 
with  H  as  the  domain. 

THEOREM  16:  The  null  space  N  of  T  is  a  closed  subspace  of  H. 

PROOF:  Let  nJt  n 2,  ...  be  any  sequence  of  vectors  in  N,  and  v  in  H  be  the  limit 
of  this  sequence.  We  wish  to  show  that  v  is  in  N.  Since  n^,  n^, . . .  are  in  N,  T(nj)  = 
T(n2)  =  . , ,  =  0.  The  limit  of  any  sequence  in  T[N]  is  in  T[N]  because  T[N]  =  0.  Since 
T  is  continuous,  T(n^)  —  T(v)  as  p  —  «o,  that  is,  T(v)  is  the  limit  of  the  sequence  T{np) 
in  T[H],  Hence  T(v)  =  0,  and  v  is  in  N.  Thus  N  is  a  closed  subspace  of  H. 
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THEOREM  17:  Let  M  denote  the  set  cf  all  vectors  in  H  which  are  orthogonal  to 
every  vector  in  N.  Then  every  vector  m  B  can  be  expressed  in  one  and  only  one  way 
as  the  linear  combination  of  a  vector  in  M  and  a  vector  in  N. 

PROOF:  Let  v  be  any  vector  in  H.  It  is  stated  without  proof  that  there  exists  a  vec¬ 
tor  n  in  N  such  that  jjv-n  jj  <5  ||  v— n  jf  for  all  n  in  N.  (The  proof  of  this  statement  is 
o  o  o 

found  in  Berberian.  )  Define  m  =  v  -  n  .  We  must  show  that  m  is  in  M,  that  is,  ni 

o  o  o  o 

is  orthogonal  to  every  vector  in  N, 

Let  n  be  any  vector  in  N,  and  X.  equal  (m^.n).  There  is  no  loss  in  generality  if 
n  is  assumed  to  be  unity,  since,  if  mQ  is  orthogonal  to  every  vector  of  unit  length 
in  N,  it  is  orthogonal  to  every  vector  in  N.  It  can  be  verified  by  direct  expansion  that 

|!mo-Xnj|2  =  jjmj2  -  jx|2. 

Now,  m  -  Xn  =  /v-n  )  -  Xn  =  v  -  (n  -Xn).  But  n  +  Xn  is  in  N;  hence, 
o  o  o  o 

I!  v“n0!l  $  !!v-{no+Xn)  jj 

from  the  choice  of  n  .  Hence 

o 

\\mj  <  ijm-Xnfj 
and  therefore 


Consequently,  jxj2  =  0,  i.  e.  (m^n)  =  0.  Thus  mo  is  in  M. 

We  see.  then,  that  any  vector  v  in  H  can  be  written  as  the  sum  of  a  vector  m  in  M 
and  a  vector  n  in  N,  We  must  show,  next,  that  for  any  vector  v,  m  and  n  are  unique. 
Assume  that  and  m2  are  in  M,  nj  and  n2  are  in  N,  and 


v  =  rn^  +  n^  =  m2  +  n 2. 


Then  +  (n^n^) 


0.  But  mj  -  m2  is  in  M  and  -  n2  is  in  N.  Taking  the 
inner  product  of  (m,-m2)  +  {n^-n^  with  (n^-n^,  we  have 


Itoj-Vf  = 


Similarly, 

II  (mj  -m2)  f  =  0. 

Hence  =  n„  and  nij  =  m2_ 

THEOREM  18:  The  quotient  space  H/N  is  a  separable  Hilbert  space,  and  the  linear 
transformation  T  from  H/N  to  the  range  of  T  is  a  continuous  transformation. 

PROOF:  To  prove  that  H/N  is  a  separable  Hilbert  space,  we  must  first  define  an 
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inner  product  in  H/N,  This  can  be  done  with  the  aid  of  the  preceding  theorem. 

Let  [v+N]  be  any  coset  in  H/N.  Since  v  can  be  written  uniquely  in  the  form  v=  m  +  n, 
the  coset  [v+N]  is  equal  to  the  coset  fm+N},  since  n  is  in  N.  Furthermore,  any  coset 
of  the  form  [m+NT]  for  some  m  is  in  H/N.  Consequently  the  set  of  cosets  of  the  form 
[m+N]  is  equal  to  the  quotient  space  H/N,  that  is,  every  coset  has  a  representative  ele¬ 
ment  in  M.  Furthermore,  for  every  coset  [vtN]  in  H/N,  there  exists  only  one  element 
m  in  M  such  that 

m  +  N  =  v  +  N, 

since  the  component  of  v  in  M  is  unique. 

Let  [m  +N]  and  [m, +N]  be  any  two  cosets  in  H/N,  and  define  the  inner  product  of 
these  cosets  as 

^[ma+N],[mb+N])  =  (ma,mb). 


It  can  be  verified  that  this  inner  product  satisfies  the  required  conditions  stated  in  Defi¬ 
nition  13.  Let  [inj+N],  [m2+N], . .  .  denote  any  sequence  in  H/N  having  the  property  that 
fl  [mm+N]-[mn+N]||  -  0  as  m.n  -  But  jj  [mm+N]-[mn+N][j  =  |mm-mnj|,  and  hence 
jjmm-mnJ!  —  0  as  m,n  —  *>.  Since  H  is  a  Hilbert  space,  there  exists  a  vector  v  in  H 
such  that  j|mn-vj|  -0  as  n-«.  In  particular,  v  is  in  M.  This  follows  directly  from 
the  statement  without  proof,  that  if  n  is  any  vector  in  N,  tnen  (m  ,  n  )  ~(v,n  }  as 
n  —  oo  (the  proof  of  this  statement  follows  from  Berberian.  }  Hence  ]|[mn+N3-[v+N3J|  —0 
as  n  —  and  thus  H/N  is  a  Hilbert  space. 

To  show  that  H/N  is  a  separable  Hilbert  space,  we  must  demonstrate  that  H/N  has 
an  orthonormal  basis.  Let  Vj,  v,, . . .  be  an  orthonormal  basis  for  H,  and  let 


v. 

1 


m.  in  M  and  n.  in  N. 
3  3 


Any  vector  m  can  be  expressed  as  a  linear  combination  of  the  m.,  since 

3 


oo  qo 

m=  I  CkVk 

1  1 


oo 


*  I  ckmk  +  Z  CkV 


But  m  can  be  written  in  only  one  way  as  the  sum  cf  a  vector  in  M  and  a  vector  in  N; 
hence,  since  m  -  in  +  o,  it  follows  that 

00 

w 

1 

Let  m'j,  ...  be  the  subsequence  of  the  sequence  mj,  m2, . . .  consisting  of  all  the 
nonzero  elements.  Then  any  vector  in  M  can  be  expressed  as  a  linear  combination  of 
the  vectors  m'j,  m!,, ....  Let 


ZQ 


m','  = 


and 


j-i 

__J _ 1  J  k  k 

I  j-1  If 

mi  -  v  m.” 

i  J  l  J  «  k 


as  dictated  by  the  Gram-Schmidt  ox'thcgonalization  procedure.  Then  the  set  m",  m£,. . . 
will  be  an  orthonormal  basis  for  M.  It  follows  in  a  straightforward  manner  that  the 
cosets  {mj*+Nj#  [mjJ+N], . . .  are  an  orthonormal  basis  for  H/N.  Hence,  H/N  is  a  sepa¬ 
rable  Hilbert  space. 

The  final  step  in  the  proof  of  this  theorem  is  to  show  that  T  is  a  continuous  trans¬ 
formation,  that  is,  we  wish  to  show  that  if 

[m^+N]  —  [m+Nj  n  — 

then 

T[mn+N}  —  T[m+N]  n  —  <*>. 

But 

T[mn+N]  =  T[mn] 

and 

Tlm+N]  =  T[m], 

and,  since  T  is  continuous,  Tfm^J  —  T{m]  n  —  «>. 

THEOREM  J  9:  Let  W  denote  the  range  of  T.  Then  W  is  a  separable  Hilbert  space. 
PROOF:  By  virtue  of  Theorem  18,  we  need  only  demonstrate  this  result  for  the  case 
in  which  T  is  invertible.  When  T  is  not  invertible  we  can  replace  H  by  H/N,  and  T 
by  T. 

Let  Wj  and  be  any  two  vectors  in  W.  Define  the  inner  product  of  two  vectors  in 
W  as 


{Wf,w2)  =  (t  1{w1),T  1{w2)). 

We  must  show  that  this  inner  product  satisfies  the  conditions  stated  in  Definition  5, 
(1)  (w1+w2,w.)  =  (t"1{w14w2),T“1(w3)^ 

=  (t‘“1{Wi)+T~1(w2),T~,(w3)) 

=  (T^WjhT-^Wj))  +  (T'^w^.T'^w,)) 

=  {Wj.WjJ  +  (w2,w3). 
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(2)  (cw,,w0)=  ( T  1  (cw , ) . T  ^v.,)) 

=  (cT'VjhT-1^)) 

=  c(r1(w1),T‘1(w2)) 

=  C(Wj,W2). 

(3)  (wrw2)  =  (t“1{w1),T"1{w2)) 

=  (t-1(w2),T“1(w1)) 

=  (w2,Wl). 

(4)  (Wj.Wj)  =  >0  if  T"l(Wj)  *  0. 

But.  T-i(w1)  *0  if  Wj  #  0,  and  hence  (Wj.Wj)  >  0  if  Wj  *  0.  Thus,  under  this  inner 
product,  W  is  an  inner  product  space.  To  show  that  W  is  a  Hilbert  space,  we  must 
verify  Definition  13. 

Let  Wj,  w2, . . . ,  wn, ...  be  a  sequence  of  vectors  in  W  having  the  property  that 

|[  wm“wn I  — 'c  as  m,n  — oo.  From  the  definition  of  the  inner  product  in  W,  ]|  wm~wn|l  ~ 

II  T~1(wm}~T~1{wn)|!.  Hence  the  sequence  T-*(wn)  in  H  also  has  the  property  that 

||T~1(wm)~T~1(wn)||  —0  as  m,n  — oo.  Consequently,  since  H  is  a  Hilbert  space,  there 

exists  a  vector  v  in  H  such  that  T  *(wn)  *"‘v  as  Since  T  is  continuous,  it  fol¬ 

lows  that  w  -*T(v)  as  n  —  <®.  Consequently,  W  is  a  Hilbert  space.  We  must  show, 
next,  that  W  has  an  orthonormal  basis. 

To  this  end,  let  Vj,  v2, _ be  an  orthonormal  basis  for  H  and  consider  the  sequence 

T(v.),  T(v  ),. . .  in  W.  The  vectors  in  this  sequence  are  orthonormal,  since 

X  Cl 

(T(v.)sT(v  j)  =  <v.,v.). 

Furthermore,  if  w  is  any  vector  in  W,  then  there  exists  a  unique  set  of  scalars  such 
that 

cc 

T-‘(w)  =  £  Vk 
1 


or 


00 

*  =  X  ckT<vk)- 
1 
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Thus,  the  set  T(v,),  T(v0),...  is  an  orthonormal  basis  for  W;  consequently,  W  is  a 

A  *— 

separable  Hilbert  space. 

In  Section  IV,  we  shall  be  concerned  with  systems  described  by  continuous  transfor¬ 
mations  for  which  the  inputs  constitute  a  separable  Hilbert  space.  The  primary  result 
there  will  be  the  der  ivation  of  a  canonic  representation  of  such  systems.  The  result 
rests  heavily  on  the  fact  that,  by  virtue  of  the  preceding  theorem,  the  set  of  system 
outputs  also  constitutes  a  separable  Hilbert  space.  This  fact,  in  conjunction  wrth  the 
following  theorem  leads  to  the  canonic  representation. 

THEOREM  20:  Lei  and  H 2  be  two  infinite-dimensional  Hilbert  spaces  having 
orthonormal  bases  Then  there  exists  an  invertible  linear  transformation  from  onto 
Hr 

PROOF:  Let  v^.v^,...  be  an  orthonormal  basis  for  Hj,  and  Wj.w^,...  be  an 
orthonormal  basis  for  H„.  Define  a  linear  transformation  T  as 


00 

T(v)  =  }  (v»V?Wk* 

T 

We  must  show  that  T  is  linear  and  invertible,  and  that  its  range  is  all  of  W.  The  line¬ 
arity  is  easily  verified.  To  show  that  it  is  invertible,  we  must  show  that  its  null  space 
is  zero,  that  is,  T(v)  -  0  implies  v  =  0,  which  follows  directly  from  the  linear  indepen¬ 
dence  of  the  orthonormal  basis  for  H 2.  To  show  that  its  range  is  all  of  W,  we  must 
show  that  if  w  is  any  vector  in  then  there  exists  a  vector  v  in  Hj  such  that 

T(v)  =  w. 

Since  H 2  has  an  orthonormal  basis,  w  can  be  expressed  in  the  form 
00 

»=  2,  <w'wk)wk- 

1 


The  sequence  =  (w.w^)  is  square  summable,  as  was  shown  in  Theorem  14.  Hence, 

00 

from  Theorem  13,  the  sum  (w.w^)  v^  has  a  limit  vector  v  in  Hj,  that  is. 


But 

00 

T{v)  =  ^  (w,wk)T{vk) 
1 


and  from  the  definition  of  T,  T(vk)  =  wk.  Hence, 
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uO 

V 

T(v)  =  (w,wk)wk  =  w 
1 

which  was  to  be  shown. 

The  result  of  Theorem  20  also  applies  when  Hj  and  are  finite  dimensional  and 
dim(Hj)  =  dim  (H2) 

in  the  Appendix  another  result  concerned  with  separable  Hilbert  spaces  having  time 
functions  as  vectors  is  derived.  This  result  plays  an  important  role  in  the  discussion 
of  Section  IV.  It  is  reserved  for  an  appendix  rather  than  developed  here  because  it 
relies  on  some  of  the  notation  and  ideas  discussed  in  Sections  III  and  IV. 

From  the  preceding  theorems  and  definitions  it  should  be  clear  that  a  separable 
Hilbert  space  is  an  inner-product  space  that  can  be  approximated  arbitrarily  closely  by 
a  finite-dimensional  inner-product  space.  If  the  domain  of  a  linear  transformation  is  a 
separable  Hilbert  space  and  the  transformation  is  cc  tinuous,  then  the  outputs  can  be 
approximated  arbitrarily  closely  by  a  finite- dimensional  approximation  to  the  input 
space.  Under  these  restrictions  on  a  linear  transformation  and  its  domain,  then,  all  of 
the  results  derived  for  finite-dimensional  spaces  can  be  extended  directly. 
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III.  GENERALIZED  SUPERPOSITION  AND  HOMOMORPHIC  SYSTEMS 
3.  1  INTRODUCTION 

In  the  past,  the  concepts  of  linear  algebra  have  been  applied  to  a  restricted  class  of 
systems,  those  that  can  be  represented  as  linear  transformations  when  vector  addition 
is  defined  as  the  sum  of  the  time  functions  in  the  vector  space  of  inputs  and  the  vector 
space  of  outputs.  Because  of  the  principle  of  superposition  which  these  systems  satisfy, 
they  are  relatively  simple  to  analyze.  This  principle  of  superposition  leads  to  charac¬ 
terization  by  means  of  the  superposition  integral.  This  representation  can  be  interpreted 
as  resulting  from  a  decomposition  of  each  of  the  inputs  into  a  linear  combination  of  infin¬ 
itesimally  narrow  pulses.  These  pulses  constitute  a  basis  for  the  vector  space  of  inputs. 
By  virtue  of  the  principle  of  superposition,  the  effect  of  the  system  on  any  function  in  the 
vector  space  of  inputs  is  determined  from  its  effect  on  the  basis  functions,  with  the 

result  that  the  input  and  output  are  related  through  the  convolution  integral. 

st 

Alternatively,  the  set  of  complex  exponentials  of  the  form  e  ,  where  s  is  a  com¬ 
plex  number,  can  be  chosen  as  the  set  of  basis  functions  for  the  input  vector  space. 
Such  functions  are  eigenfunctions  of  linear  time-invariant  systems,  and  hence  such  sys¬ 
tems  have  the  effect  of  multiplying  these  functions  by  constants.  Thus,  when  complex 
exponentials  are  used  as  a  basis  for  the  vector  space  of  inputs  to  a  linear,  time- invariant 
system  the  system  is  described  by  the  set  of  constants  by  which  it  attenuates  these  expo¬ 
nentials,  that  is,  it  is  described  by  its  system  function. 

We  have  noted  that  vector  spaces  of  time  functions  could  be  constructed  with  a  variety 
of  definitions  for  vector  addition.  When  advantage  is  taken  of  the  generality  afforded  by 
the  postulates  of  vector  addition  and  scalar  multiplication,  systems  that  are  generally 
considered  to  be  nonlinear  can  be  represented  as  linear  transformations  between  vector 
spaces.  Formulated  in  terms  of  system  theory,  this  procedure  leads  to  a  generalization 
of  the  principle  of  superposition.  This  principle  encompasses  the  class  of  linear  sys¬ 
tems,  as  well  as  many  nonlinear  systems.  In  particular,  it  encompasses  all  invertible 
systems,  as  well  as  many  systems  that  are  not  invertible. 

3.2  GENERALIZED  PRINCIPLE  OF  SUPERPOSITION 

A  linear  system  with  transformation  T  is  characterized  by  the  property  that  if  v^(t) 
and  v.,(t}  are  any  two  system  inputs,  then 

T[c1v1(t)+c2v2(t)3  =  CjTjVjR}]  +  c2T[v2(t)] 

for  any  scalars  Cj  and  c2.  From  this  definition,  it  is  clear  that  the  transformation  <j>, 
defined  as 

♦(vwl  =  ev(t)  (1) 

is  nonlinear,  since 
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(2) 


4>  [cjVj {t)+c2v2(t)3  =  [^vpf1  [*(v2)f2. 

The  transformation  of  Eq.  1  does  obey  a  form  of  superposition  in  the  sense  that  its 
response  to  the  linear  combination  of  a  set  of  inputs  is  determined  by  its  response  to 
each  of  the  inputs  m  the  set.  The  manner  in  which  the  individual  responses  are  combined 
to  produce  the  response  to  a  linear  combination  of  inputs  io  defined  by  the  right-hand  side 
of  Eq.  2. 

If  the  set  of  inputs  to  the  system  of  Eq.  1  constitutes  a  vector  space  under  addition, 
then  the  set  of  outputs  constitutes  a  vector  space  under  multiplication  and  the  transfor¬ 
mation  of  Eq.  1  is  an  algebraically  linear  transformation  between  these  spaces.  Thus 
this  system  falls  within  the  framework  of  linear  algebra. 

In  order  to  avoid  confusion  with  the  more  conventional  notion  of  a  linear  system,  sys¬ 
tems  that  are  algebraically  linear  will  be  referred  to  as  homomorphic  systems,  since 
they  are  represented  by  homomorphic  transformations  between  vector  spaces.  The  vec¬ 
tor  space  of  inputs  to  a  homomorphic  system  will  be  denoted  V,  and  the  vector  space  of 
outputs  will  be  .j,.oted  W.  The  vector  addition  of  two  inputs  v^  and  v2  will  be  denoted 
Vj  o  v^,  and  the  multiplication  of  an  input  vector  v  with  a  scalar  c  will  be  expressed  as 
(c^v) .  The  operations  o  and  >  will  be  referred  to  as  the  input  operations  of  the  homo¬ 
morphic  system.  The  vector  addition  of  two  outputs  and  w2  will  be  denoted 
WjO  w2,  and  the  multiplication  of  an  output  vector  w  by  a  scalar  c  will  be  expressed 
as  (c/w).  The  operations  a  and  /  will  be  referred  to  as  the  output  operations  of  the 
system. 

A  homomorphic  system  with  system  transformation  <J>  has  the  property  that 

4»[(c1>v1)o{c2>v2)]  =  [Cj/^Vj)]  o  [c2/(j>(v2)j  (3) 


for  any  inputs  Vj  and  v2  and  any  scalars  Cj  and  c2.  The  property  described  by  Eq.  3 
will  be  referred  ♦.  i  as  the  generalized  principle  of  superposition.  In  the  particular  case 
for  which  the  operations  o  and  o  are  chosen  to  be  addition  and  the  operations  >  and  / 
are  chosen  to  be  multiplication,  Eq.  3  reduces  to  the  principle  of  superposition  as  it 
applies  to  linear  systems. 

A  homomorphic  system  with  system  transformation  <j>,  input  operation  o  and  output 
operation  o  ,  will  be  denoted  as  shown  in  Fig.  1 .  An  example  of  a  homomorphic  system 
is  that  system  having  the  transformation  specified  by  Eq.  1.  In  this  case,  the  operations 

o,  >,  o  ,  and  /  are  chosen  as 

* 


v.  o  V. 


=  v  +  v 
1  2 


c>v  =  cv 


w 


1 


°  w2  =  W1W2 


c/w  =  (w)C. 
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This  system  is  thus  represented  as  shown  in  Fig.  2. 


;  0 

r— — » 

°  + 

• 

A 

^  W  v  « 

w  =  e  v 

Fig.  1.  Representation  of  a  homomor-  Fig.  2.  Example  of  a  homomorphic  sys- 

ph.'-:  rvjtem  with  input  operation  tem  with  addition  as  the  input 

G-  output  operation  a  ,  and  ays-  operation  and  multiplication  as 

t*  ■"  transformation  c .  the  output  operation. 

Since  homomorphic  systems  are  represented  by  linear  transformations  between  vec¬ 
tor  spaces,  algebraic  theorems  applying  to  such  transformations  can  be  reformulated 
in  terms  of  homomorphic  systems.  One  of  the  objectives  here  will  be  to  discuss  the 
application  of  these  theorems  to  homomorphic  systems. 

3.3  INVERTIBLE  HOMOMORPHIC  SYSTEMS 

THEOREM  21:  Let  4>  be  the  system  transformation  of  an  invertible  system  (a  sys¬ 
tem  for  which  the  system  transformation  is  invertibie)  for  which  the  inputs  constitute 
a  vector  space  V  with  operations  o  and  >.  Then  the  system  can  be  homomorphic  under, 
at  most,  one  choice  for  the  output  operations. 

PROOF:  We  wish  to  show  that  if  the  system  is  homomorphic,  then  the  output  oper¬ 
ations  are  unique.  Assume  that  there  are  two  sets  of  output  operations  under  which  the 
system  is  homomorphic.  Let  the  first  set  be  denoted  o  and  /,  and  the  second  set  be 
denoted  ■  and  J~ .  If  Wj  and  w,,  are  any  two  outputs,  then  we  wish  to  show  that 
c/ w  =  c _/~v/  for  all  scalars  c  in  the  field  F  and  all  outputs  w. 

If  we  denote  the  inverse  of  «}»  by  4>—  ,  then  if  4>  (v)  =  w  it  follows  that  v  =  <{>  (w).  By 

assumption,  the  system  4>  is  homomorphic  with  input  operations  o  and  >  and  output 
operations  o  and  /.  By  virtue  of  Theorem  8,  the  inverse  system,  having  system  trans¬ 
formation  4>-i,  is  homomorphic  with  input  operations  o  and  /  and  output  operations  o 
and  >.  Hence  if  Wj  and  w^  are  any  two  outputs  and  c  is  any  scalar  in  F,  we  have 

4>  ^Wj  o  w2]  =  4>_1{Wj)  o  <j>  1  (w,)  (4) 

and 

4*~"^[c/Wj]  =  c>4>~1{Wj).  (5) 

By  assumption,  however,  4>  is  also  homomorphic  with  input  operations  o  and  >  and 
output  operations  ■  and  J~ .  Hence  4>  is  homomorphic  with  input  operations  ■  and  J~ 
and  output  operations  o  and  > .  Consequently, 
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(6) 


_t  .  __  1 

4>  *[Wj  ■  w,]  =  <j>  i(wI)  o  <j>  a(w2) 

and 

rHc/Wj]:  C^fWj).  (7) 

Comparing  Eqs.  4  and  6,  we  have 

r^Wj  o  w2]  =  4>_1[wj  ■  w23.  (8) 

If  the  vector  in  V  given  in  Eq.  8  is  put  through  the  system  4>»  we  have 

Wj  o  w2  =  Wj  ■  w2  (9) 

for  all  outputs  Wj  and  w.,.  Similarly,  comparing  Eqs.  5  and  7,  we  find 


4>  ^c/wj]  =  <}>  '[c/wj] 


or 

c/wj  =  c/wj 


(10) 


(ID 


for  all  outputs  Wj  and  all  scalars  c. 

THEOREM  22:  Let  4>  be  an  invertible  system  the  inputs  to  which  constitute  a  vector 
space  under  the  operations  o  and  > .  Then  there  always  exists  a  unique  set  of  output 
operations  under  which  the  set  of  outputs  constitutes  a  vector  space,  and  the  system  is 
homomorphic. 

PROOF:  Let  W  denote  the  set  of  outputs  of  <j>.  If  Wj  and  w2  are  any  two  outputs, 
let  their  vector  sum  be  defined  as 


Wj  o  w 


2  =  ^jV^fWj)  o  <J>-1(w2) 


(12) 


Similarly,  let  the  scalar  multiplication  of  any  output  w  by  a  scalar  c  be  defined  as 


c/w  =  <j>[c>4>  X(w)]. 


(13) 


We  need  to  verify  only  that  under  this  choice  of  output  operations,  the  system  4>  is 
homomorphic.  Theorem  6  then  requires  that  the  set  of  outputs  constitutes  a  vector 
space  under  these  operations. 

Let  Vj  and  v2  be  any  two  inputs,  and  c^  and  c^  be  any  two  scalars.  Then  4>  is 
homomorphic  if 

4>[<C1  >V1>  O  (c2>v2)]  =  [Cj/cMVj)}  o  [c2/4>(v-2)3.  (14) 

Evaluating  the  right-hand  side  of  Eq.  14  according  to  Eqs.  12  and  13,  we  have 
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r  .  .  ”s  r  .  . 

[c ^(Vj) ]  a  [c2/4>(vz)]  =  |  4*(c1><i»-14»<v1)^  j  a  |  <t>(c2><J>_1<i>(v2)^  ! 


[*Mc j  >v j ) ]  □  [<j>(c2>v2)] 


=  4> 


(♦“^(c^Vj))  o  (d>-14)(c2>v2)jj 


or 


[c1/<fr{vJ )]  a  [cz/<t»(v2)3  =  o  (c2>v?)].  (15) 

Hence,  4>  is  a  homomorphic  system. 

Furthermore,  for  a  specified  input  space  the  set  of  output  operations  under  which 
4>  is  homomorphic  is  unique.  This  is  seen  by  a  direct  application  of  Theorem  21. 

Theorem  2?  states  that  the  class  of  homomorphic  systems  includes  all  invertible 
systems.  “When  the  input  operations  and  the  system  transformation  are  specified,  the 
output  operations  are  given  uniquely  by  Eqs.  12  and  13.  In  applying  the  theory  of 
homomorphic  systems,  it  would  not  be  expected  that  the  output  operations  would  be  con¬ 
structed  by  means  of  Eqs.  12  and  13,  since  this  would  require  a  precise  characteri¬ 
zation  of  the  system  transformation.  Because  of  the  uniqueness  of  the  output  operations, 
however,  we  know  that  no  matter  how  these  operations  are  obtained,  they  must  satisfy 
Eqs.  12  and  13.  Equations  12  and  13  allow  us  to  construct  examples  of  homomorphic 
systems  as  an  aid  to  developing  the  theory.  By  virtue  of  the  uniqueness  of  the  output 
operations,  examples  constructed  in  this  way  will  not  depend  on  a  restricted  choice  for 
the  output  operations  of  the  system. 

As  an  example  of  the  application  of  Eqs.  12  and  13,  consider  a  linear,  invertible, 
time-invariant  system.  Let  h(t)  denote  the  impulse  response  of  the  system,  and  h(t) 
the  impulse  response  of  the  inverse  system.  Let  the  set  of  inputs  V  be  a  vector  space 
under  addition.  Since  the  system  was  specified  to  be  linear,  we  know,  without  application 
of  Eqs.  12  and  13  that  the  system  is  homomorphic  if  vector  addition  of  the  outputs  is 
chosen  as  the  sum  of  the  time  functions,  and  scalar  multiplication  on  the  output  set  is 
chosen  as  the  product  of  the  time  function  and  the  scalar. 

Since  this  set  oi  output  operations  is  unique  under  the  specified  choice  of  input  oper¬ 
ations,  application  of  Eqs.  12  and  13  must  yield  this  result.  Specifically,  Eq.  12 
requires  that  if  w^  and  w2  are  any  two  outputs,  then 


Wj  o  w2  =  h(t)  ®[h(t)  ®w1(t)+h(t)  ®w?(t)],  (16) 

where  ®  denotes  convolution.  Expanding  Eq.  16, 

Wj  o  w2  =  h(t)  ®h(t)  ®Wj(t)  +  h(t)  <8>h(t)  ®w2(t), 
since  convolution  distributes  over  addition.  The  convolution  of  h(t)  and  h(t)  is  the  unit 
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impulse  uQ(t);  hence, 

Wj  O  w,  =  uo(t)  0  Wj  (t)  v  uo(t)  0  w2(t) 
or 

Wj  O  VV‘2  =  Wj{t)  +  w?(t). 

Similarly,  Eq.  13  requires  that 

c/w (t)  =  h(t)  0  [c(h(t)0w(t))] 
or 

e/w(t)  -  c[h(t)0  h(t>  0v/(t)] 
and  hence 

c/w(t)  =  cw(t). 

As  another  example  of  the  application  of  Eqs.  12  and  13,  consider  the  system  having 
the  transformation  4*  given  by 

w  =  4>(v)  =  ev.  (17) 

The  transformation  corresponding  to  the  inverse  system  is 

v  =  $  *  (w)  =  Inw.  (18) 

[f  the  set  of  inputs  is  chosen  as  a  vector  space  under  addition  with  the  field  chosen  as 
the  field  of  real  numbers,  then  application  of  Eq,  12  requires  that 

W1  a  W2  =  exPtlnwj+lnw2^ 

or 

Wj  o  w2  =  WjW2  (19) 

and  Eq.  13  requires  that 
c/w  =  exp[c  InWj] 

or 

c/w  =  [wj]0,  (20) 

The  transformation  of  Eq.  17,  however,  is  an  invertible  transformation  under  any  input 
vector  space.  If  we  choose  as  the  set  of  inputs,  a  vector  space  over  the  field  of  real 
numbers  with  vector  addition  defined  as  the  product  of  the  time  functions  and  scalar 
multiplication  defined  as  the  time  function  raised  to  the  scalar  power,  then  the  set  of 
output  operations  corresponding  to  the  transformation  of  Eq.  17  will  be  different  from 
those  given  in  Eqs.  19  and  20.  These  output  operations  can,  however,  be  determined 
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through  the  use  of  Eqs.  12  and  13.  Applying  Eq.  12,  the  vector  sum  of  any  two  outputs 
Wj  and  w^  is  given  by 

[in w.  Inw^] 

Wj  o  =  e 


or 


lnw, 
f  lnw  1  2 


W1  D  w?  =  Le 


Similarly/,  scalar  multiplication  in  the  set  of  outputs  is  given  by 
flnwjc 


c/w  =  el 


3.4  NONINVERTIBLE  HOMOMORPHIC  SYSTEMS 

Theorem  22  guarantees  that  a  system  that  is  invertible  for  an  input  vector  space  V, 
is  homomorphic  for  this  set  of  inputs.  When  a  system  is  not  invertible  for  a  given  set 
of  inputs  we  are  no  longer  assured  that  the  system  will  be  homomorphic  with  respect  to 
these  inputs.  Theorems  6  and  12,  together  with  the  properties  of  quotient  spaces,  lead 
tc  necessary  and  sufficient  conditions  on  the  system  transformation  and  the  set  of  inputs 
associated  with  a  given  system  such  that  the  system  is  homomorphic.  Before  proceeding 
to  a  statement  of  the  conditions  in  terms  of  a  theorem,  it  would  be  well  to  indicate  the 
direction  which  we  take  in  this  theorem. 

Consider  a  system  with  transformation  4»  and  input  vector  space  V.  By  choosing 
any  subspace  N  of  V,  V  can  be  divided  into  cosets.  It  is  necessary  that  these  cosets 
be  formed  with  respect  to  a  subspace  of  V  rather  that  with  respect  to  any  arbitrary  set 
of  vectors  in  V,  in  order  that  these  cosets  be  well  defined.  If  N  is  the  null  space  of 
the  system,  each  vector  in  a  given  coset  will  result  in  the  same  output.  Furthermore, 
it  will  be  seen  in  Theorem  23  that  if  each  vector  in  a  given  set  of  vectors  in  V  maps 
onto  the  same  vector  and  if  the  system  is  homomorphic,  then  this  set  of  vectors  must 
be  an  element  in  the  quotient  space  V/N.  In  particular,  any  coset  in  V/N  which  does 
not  map  onto  the  identity  element  in  the  vector  space  of  outputs  cannot  be  a  subspace 
of  V. 

The  approach  to  deriving  necessary  and  sufficient  conditions  on  a  system  in  such  a 
way  that  it  be  homomorphic,  will  be  based  on  a  consideration  of  conditions  under  which 
an  invertible  transformation  can  be  defined  from  a  quotient  space  associated  with  V,  to 
the  set  of  outputs  of  the  system.  If  such  an  invertible  transformation  can  be  defined, 
then  Theorem  22  can  be  employed  to  determine  a  set  of  output  operations  under  which 
the  system  is  homomorphic. 

THEOREM  23:  Let  be  the  transformation  of  a  system  with  an  input  vector  space 
V.  Let  Sj,  S2>  ....  SR  be  subsets  of  V  with  the  properties  that 
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{1}  All  elements  in  a  given  set  S.  produce  the  same  output.  This  output  corre- 
spending  to  a  set  will  he  denoted 

(2)  If  4>  [S.j  is  the  output  produced  by  the  set  of  inputs  and  J  is  the  output 

produced  by  the  set  S.,  all  i  and  j,  then  if’IS,]  =  4>[S.j  implies  that  i  ~  j. 

J  *  J 

Then,  at  most,  one  of  the  sets  S^,  . S n  can  be  a  subspace  of  V. 

PROOF:  Properties  (1)  and  (2)  imply  that  no  two  sets  and  have  an  element  in 
common.  For,  assume  that  s  is  m  S,  and  s  is  in  S..,  then  from  property  (I), 


4>[SiJ  =  <j>(s) 


and 

<J>[Sj]  =  <Ms), 
and  hence 

which  by  virtue  of  property  (2)  requires  that  i  =  j. 

Any  subspace  of  V  must  contain  the  identity  element  in  V.  Hence  if  two  of  the  sets 
Sj,  S2,  ,  Sn  are  subspaces  of  V,  they  must  both  contain  the  identity  element,  and 

hence  would  have  an  element  in  common.  Thus,  at  most,  one  of  the  sets  S^,  S2,  ....  Sr 
can  be  a  subspace  of  V. 

THEOREM  24:  Let  <j>  be  the  transformation  of  a  system  with  an  input  vector  space 
V.  Let  Sj,  S2,  . . .  ,  Sn  be  subsets  of  V  with  the  properties  that 

(1)  All  elements  in  a  given  set  produce  the  same  output. 

(2)  If  v  is  a  vector  in  V,  then  4>{v)  -  <j>[Sjj  implies  that  v  is  in  S^, 

{3}  The  sets  are  distinct;  that  is,  if  *[S^J  =  4>[S  },  then  i  =  j. 

(4)  The  union  of  the  sets  S. ,  ,  ....  S  is  all  of  V. 

i  h  n 

If  none  of  the  sets  Sj,  S2>  . . . ,  is  a  subspace  of  V,  then  the  system  cannot  be 
homomorphic. 

PROOF:  Let  e  denote  the  identity  element  in  V  and  lei  Sg  denote  the  set  of  ele¬ 
ments  such  that 

♦(e)  =  <MSeL 

If  4>  is  homomorphic,  then  the  element  4>(e}  must  be  the  identity  element  in  the  vector 
space  of  outputs  W  by  virtue  of  Theorem  4.  Because  of  property  (2)  imposed  on  the  sets 

Sj ,  S2, _ _  Sn,  the  set  Sg  contains  all  of  the  elements  in  V  which  map  through  4>  to 

the  identity  in  W.  Hence,  by  definition,  Sg  is  the  null  space  of  V  and  Theorem  6 
then  requires  that  Se  be  a  subspace  of  V. 

Property  (4)  imposed  on  the  sets  Sj,  S2> _ _  Sn  requires  that  this  set  of  subsets  of 

V  must  contain  the  subset  of  all  elements  of  V  that  map  onto  4>(e)  and  hence  must  contain 
S  .  If  4>  is  homomorphic,  then  one  of  these  subsets  must  be  a  subspace  of  V;  hence,  if 
none  of  these  subsets  is  a  subspace  cf  V,  then  the  system  cannot  be  homomorphic. 
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Theorems  23  and  24  together  require  that  a  necessary  condition  so  that  a  system 
be  homomorphic  is  that  we  can  find  one  and  no  more  than  one  subspace  in  V  which  con¬ 
tains  all  of  the  elements  that  map  onto  a  single  element  in  the  set  of  outputs.  In  general, 
if  the  system  transformation  is  well  defined,  then  we  are  guaranteed  that  no  more  than 
one  such  subspace  can  be  found.  If  one  such  subspace  exists,  then  it  must  contain  the 
identity  element  frem  V.  Hence,  if  e  denotes  the  identity  element  in  V,  then  Theorem 
24  can  be  reformulated  to  require  that  a  system  with  transformation  4»  cannot  be  homo¬ 
morphic  unless  the  set  of  ail  elements  v  in  V  such  that  4>(v)  =  4>(e)  is  a  subspace  of  V. 
As  shown  in  Theorem  25,  this  condition,  together  with  one  other  condition,  provides 
necessary  and  sufficient  conditions  on  a  system  so  that  it  will  be  homomorphic. 

THEOREM  25:  Let  <$>  be  the  system  transformation  of  a  system  with  input  vector 
space  V.  Then  necessary  and  sufficient  conditions  so  that  the  system  be  homomorphic 
are 


(1)  The  set  of  elements  S  in  V  is  a  subspace  of  V,  where  S  is  the  set  of  all  ele¬ 
ments  s  in  V  with  the  property  that  4>(s)  =  4>(e),  where  e  is  the  identity  m  V. 

(2)  Consider  the  quotient  space  V/S&,  where  is  a  subspace  of  V.  Let  v  o  S& 
be  any  coset  in  the  quotient  space.  If  v'  is  any  element  in  voS^,  then 
4>(v')  =  4>(v).  (This  condition  requires  that  any  given  coset  in  the  quotient  space 
V/S^  have  a  unique  mapping  in  W,  that  is,  d[v  o  Sg]  will  be  well  defined  for 
every  v  in  V,  and  each  coset  will  produce  a  different  output. ) 

PROOF:  To  prove  that  these  two  conditions  are  necessary,  we  assume  that  the  sys¬ 
tem  is  homomorphic  and  show  that  this  requires  that  conditions  {1)  and  (2)  hold. 

Assume  that  the  system  is  homomorphic.  Then,  by  definition,  Sg  is  the  null  space 
of  <)>.  Theorem  6  then  requires  that  be  a  subspace  of  V.  To  show  that  condition  (2) 
is  implied  by  the  assumption  that  the  system  is  homomorphic,  let  W  denote  the  output 
vector  space  with  vector  addition  and  scalar  multiplication  denoted  by  a  and  /,  respec¬ 
tively.  If  e  is  the  identity  in  V,  then  4>{e)  is  the  identity  in  W.  Let  voSg  be  any  coset 

in  V/S  ,  and  v'  be  an  element  in  v  o  S  .  Then  there  exists  some  element  s  of  S  such 
e  e  e 

that 


v*  =  v  o  s. 


Hence  <J>{v)  =  <{;{v  os)  -  4>(vj  o  4>(s).  But,  since  s  is  in  Se>  4>{s)  is  the  identity  element 
in  W.  Consequently, 

=  4>{v)  all  v*  in  v  o  S  , 

and  therefore  condition  (2)  holds. 

To  show  that  conditions  (1)  and  (2)  are  sufficient,  we  shall  assume  that  these  con¬ 
ditions  hold,  and  prove  that  the  system  must  then  be  homomorphic.  Condition  (l)  per¬ 
mits  the  construction  of  a  quotient  space  with  respect  to  the  set  Se,  since  it  requires 
that  this  set  be  a  subspace  of  V.  Condition  (2)  then  provides  an  invertible  transfor¬ 
mation  between  the  quotient  space  V/S  and  the  set  of  outputs  W,  since  for  any  coset 
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v  o  Se  in  the  quotient  space  V/S  there  exists  a  well-defined  mapping  to  an  element  in 
W  given  by 

%[v  o  Sj  s  4»i v> . 

The  transformation  <t>  can  then  be  considered  as  the  system  transformation  associated 
with  an  invertible  system  with  inputs  that  are  elements  in  the  quotient  space  V/S  . 
Theorem  22  requires  that  the  system  with  transformation  £  be  a  homomorphic  system 
and  that  the  set  of  outputs  W  be  a  vector  space.  Specifically,  vector  addition  in  the  set 
of  outputs  is  defined  by 

Wj  O  w2  =  J  ?_1(Wj)  o$_1(w2)lf  (21) 

__  _5 

^•1  ^ j 

where  and  w2  are  any  two  elements  in  W.  The  elements  4>  (w^)  and  4>  (w2)  are 

cosets  in  the  quotient  space  V/S  .  Scalar  multiplication  in  W  is  defined  as 

c/w  a  £  [c>£_1(w}3.  (22) 

It  remains  only  to  show  that  with  this  choice  of  output  operations  the  system  with  system 
transformation  4>  and  input  vector  space  V,  is  homomorphic.  Let  v  and  v,  be  any  two 

1  L 

elements  in  V.  Then 

4>(Vj)  =  [vj  o  Se], 

4><v2)  =  ^[v2  o  Se], 

4*(vj  o  v2)  =  ^[(Vj  o  v2)  o  Se]. 

From  the  definition  of  vector  addition  in  the  quotient  space  V/Se  as  stated  in  Theorem 
3,  however, 

(Vj  o  v,)  o  Sg  =  (v}  o  Sg)  o  {v2  o  Se>; 
hence, 

4»{v1  o  v2)  =  6[(v1  o  Se)  o  (v2  o  Se)]. 

Since  4>  is  homomorphic, 

£[(v,  o  Se>  o  (v2  o  Se)]  =  ^[vj  o  Se3  o^[v2o  Se3. 

By  definition  of  4>,  however, 
o  Se3  =  4»(Vj} 
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£[v2  o  Se]  =  4>{v2L 
Consequently, 

°  v2*  ~  ^V1  0  Se^  0  4>[v2  o  Sg]  -■  4»{v-j}  a  <i>(v2).  (23) 

Next,  let  v  be  any  element  in  V,  and  c  any  scalar  in  the  field.  Then 
4>(v)  =  £[v  o  Sg] 

{t*(c>v)  =  4>  [(c>v)  o  SeL 

From  the  definition  of  scalar  multiplication  in  V/Se>  however, 

(c>v)  o  Sg  =  c>(v  o  Se); 

hence, 

<J»(c>v)  =  £[c>(v  o  Se)]. 

But,  since  <}>  is  homomorphic, 

£[c>(v  o  Se)]  =  c/  4>  (v  o  Se) 


or 

$  [c>(v  o  Se)]  =  c/4>(v) 

Therefore 

4>(c>v)  =  c/4>(v).  (24) 

Equations  23  and  24  are  sufficient  for  the  system  to  be  homomorphic. 

THEOREM  26:  Let  <$>  be  a  homomorphic  sy st  with  input  vector  space  V  and  null 

space  N.  Let  the  subset  S  of  V  be  defined  by  the  properties 

(1)  If  Sj  and  s2  are  any  two  elements  in  S,  then  4>{Sj)  =  4>(s2). 

(2)  If  s  is  in  S  and  v  is  in  V,  then  <j>(s)  =  <$>{v)  implies  that  v  is  in  S, 

Then  each  such  subset  is  a  distinct  coset  in  V/N.  In  particular,  each  subset  S  is  the 
coset  s  o  N,  where  s  is  any  element  in  S. 

PROOF:  Let  S  be  any  subset  of  V  having  properties  (1)  and  (2).  Consider  the 
coset  s  o  N,  where  s  is  in  S.  Let  s  o  n  be  any  element  in  this  coset.  Then  <$>(s  o  n)  = 
4>{s)  o  <j>(n) ,  since  4>  is  homomorphic.  But  $(n)  is  the  identity  in  the  output  set  W,  since 
n  is  in  the  null  space.  Hence 

4»<s  o  n)  =  ${s); 

consequently,  s  o  n  is  ir  S  by  virtue  of  property  (2).  Thus  the  coset  s  o  N  is  contained 
in  the  set  S. 
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element  in  3.  Then  Sj  can  be  expressed  as 


Sj  =  so 


(s"1  o  sL). 


-1 


But  s  o  Sj  is  in  the  null  space  N,  since 


4>(s)  -  <t>(s j) , 


and  the  system  is  homomorphic.  Hence  for  any  s^  in  Sf  there  exists  an  n1  in  N  such 
that 


Sj  =  s  o  n^; 

consequently,  s^  is  in  the  coset  s  o  N.  Thus  S  is  contained  in  s  o  N  and  therefore 
S  =  s  o  N. 

The  fact  that  each  of  these  cosets  is  distinct  follows  directly  from  property  (2)  and 
Theorem  4. 

Theorems  25  and  26  provide  a  procedure  for  investigating  whether  or  not  a  given 
system  is  homomorphic.  Specifically,  according  to  Theorem  25,  we  first  find  all  ele¬ 
ments  in  the  input-vector  space  V  which  produce  the  same  output  as  inat  produced  by  the 
identity  element  in  V.  If  this  set  is  not  a  subspace  of  V,  then  the  system  cannot  be 
homomorphic.  If  it  is  a  subspace  of  V,  we  must  then  determine  whether  or  not  every 
coset  constructed  by  adjoining  each  vector  in  V  to  the  null  space  has  a  unique  output. 
If  each  of  these  cosets  has  a  unique  mapping,  then  the  system  is  homomorphic.  If  not, 
the  system  cannot  be  homomorphic.  Alternatively,  the  second  condition  can  be  refor¬ 
mulated  in  terms  of  Theorem  25.  The  input  vector  space  can  be  divided  into  all  of  those 
sets  that  produce  unique  outputs  in  W.  The  system  is  only  homomorphic  if  each  of  these 
sets  is  a  coset  in  the  space  V/S  ,  where  Sg  is  the  subspace  of  V  defined  previously. 

As  an  example  of  the  application  of  this  procedure,  consider  an  invertible  system 
with  input  vector  space  V  and  system  transformation  $.  Since  the  system  is  invertible, 
the  set  Se  contains  only  the  identity  in  V.  The  identity  alone  constitutes  a  subspace; 
hence,  the  first  condition  of  Theorem  25  is  met.  Each  of  the  cosets  in  the  quotient 
space  V/S  when  S  is  the  identity  alone  contain  only  a  single  vector;  specifically,  the 
coset  voSg  contains  only  the  vector  v.  Thus  all  of  the  vectors  in  any  given  coset  pro¬ 
duce  the  same  output  and  each  coset  is  associated  with  a  different  output.  Consequently 
the  second  condition  of  Theorem  25  is  satisfied.  Alternatively,  if  we  interpret  the 
second  condition  of  Theorem  25  by  means  of  Theorem  26,  we  can  divide  V  into  sets 
defined  by  the  property  that  any  given  set  contains  all  of  the  inputs  that  produce  a  given 
output.  Again,  since  the  system  is  invertible,  each  of  these  sets  contains  a  single 
vector  from  V.  These  are  the  cosets  in  V/S  ,  and  hence  by  this  interpretation,  the 
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of  the  identity  alone,  which  by  Theorem  10  is  a  necessary  and  sufficient  condition  for  a 
homomorphic  system  to  be  invertible,  Eqs.  21  and  22  reduce  to  Eqs.  12  and  13.  It 
follows,  then,  that  as  we  have  already  shown  by  other  means,  any  invertible  system  is 
homomorphic. 

As  another  example  of  the  application  of  Theorem  25,  consider  a  system  with  trans¬ 
formation  <j>  defined  as 


<Mv)  =  v2. 

Let  us  choose  the  input  vector  space  V  to  be  a  space  under  addition  and  require  tha'.  it 
contain  more  than  just  the  identity  element.  Since  the  identity  element  in  V  is  the  ele¬ 
ment  0,  the  set  Sg  is  the  set  of  all  elements  s  in  V  with  the  property  that 


•Ms)  =  4>(0)  =  o. 

From  Eq.  25  we  see  that  there  is  no  element  in  V  other  than  the  identity  which  will 
produce  zero  output;  hence,  the  set  S  contains  the  identity  alone.  The  identity  is  a 
subspace  of  V;  hence,  the  first  condition  of  Theorem  26  is  met.  Because  the  set  S 
is  the  identity  alone,  the  coset  v  o  Sg  in  V/Se  vector  v-  But  the  coset  voSg 

and  (-v)  o  Se  produces  the  same  output;  hence,  condition  {2)  of  Theorem  26  is  not  met. 
Thus  the  given  system  is  not  homomorphic. 

As  a  third  example,  consider  the  system  with  transformation  <{>  defined  by 

*<V>=7  3T  (25) 


Let.  the  input  vector  space  be  chosen  as  a  space  under  multiplication,  in  which  case  the 
identity  e  in  V  is  e  =  1.  The  set  Se  is  then  the  set  of  all  elements  s  in  V  such  that 


1  -d-§  =  o 

s  dt  ’ 


which  is  the  set  of  ail  constant  inputs  in  V.  It  can  be  verified  that  this  set  is  a  subspace 
of  V.  Now,  consider  any  et  v  o  Se>  We  must  show  that  all  elements  of  the  form 
V'C,  where  c  is  a  constant,  produce  the  same  output,  for  any  c.  But 


4>(v-c)  = 


v*c 


d 

m 


<v*c)  =  i  3F 


and  hence  4>{v-c)  is  independent  of  c,  as  required.  We  must  then  ascertain  that  each 
distinct  coset  is  associated  with  a  different  output.  In  Theorem  4  we  noted  that  if  two 
cosets  have  any  element  in  common,  they  are  identical  cosets.  Conversely,  if  two 
coseis  differ  by  at  least  one  element,  then  they  will  have  no  elements  in  common.  Con¬ 
sider  two  distinct  cosets  v,  o  S  and  v,  oS  .  Then 

1  e  2  e 


♦Ivi  o  S J  =  X 


dvl 

"a? 


d 

at 


[lnvj] 
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and 


v2  °  Se] 


v„ 


dV  rl 

~d T  =  dt  ^lnv2^ 


The  system  witn  the  transformation  of  Eq.  25  will  be  homomorphic  if  and  only  if 

4»(v i  o  Sg  =  <j>[v2  o  Sg]  (26) 

implies  that  oS^  and  o  Sg  are  identical  cosets,  i.  e.  that  is  a  scalar  multiple 
of  v^.  But  Eq.  26  requires  that 


^-Unv^^  ^-[lnv2] 


or 

In  Vj  =  In  v2  +  c 
for  some  constant  c.  Hence 

V1  =  CV2  f27) 

for  some  constant  c';  therefore,  we  conclude  that  the  system  is  homomorphic. 

In  Theorem  22  we  investigated  the  necessary  and  sufficient  conditions  on  an  invert¬ 
ible  system  such  that  a  set  of  output  operations  could  be  selected  to  make  the  system 
homomorphic.  These  output  operations  were  shown  to  be  unique.  Similarly,  in  Theo¬ 
rem  25  necessary  and  sufficient  conditions  on  a  noninvertible  system  were  given  so  that 
the  system  was  homomorphic.  It  was  shown  that  when  these  conditions  were  met,  and 
the  output  operations  were  chosen  according  to  Eqs.  21  and  22,  then  the  system  would  be 
homomorphic.  As  for  an  invertible  system,  it  can  be  shown  that  the  output  operations 
chosen  according  to  Eqs.  21  and  22  are  a  unique  choice. 

THEOREM  27:  Let  <j>  be  the  system  transformation  of  a  (not  necessarily  invertible) 
system,  with  input  vector  space  V.  Then  there  is,  at  most,  one  choice  for  the  output 
operations  under  which  the  system  is  homomorphic. 

PROOF:  The  desired  result  can  be  shown  by  assuming  that  the  system  is  homomor¬ 
phic  under  more  than  one  choice  for  the  output  operations,  and  from  this  deriving  a  con¬ 
tradiction. 

Specifically,  let  W  denote  the  set  of  outputs.  Let  Nj  denote  the  null  space  of  the 
system  when  the  output  operations  are  a  and  /,  and  let  N2  denote  the  null  space  of  the 
system  when  the  output  operations  are  ■  and  _/*•  If  e  denotes  the  identity  in  V,  then 
Nj  contains  all  of  the  elements  n  in  V,  with  the  property  that  $(n)  =  4>(e).  Similarly, 

N2  contains  all  of  the  elements  n'  in  V  such  that  4>(n')  =  4>(e).  Hence,  Nj  =  N2«  that  is, 
the  null  spaces  under  the  two  choices  for  the  output  operation  are  identical.  Let  Nj  and 
both  be  denoted  by  N,  and  consider  the  quotient  space  V/N.  Theorem  12  requires 
that  the  system  having  the  space  V/N  as  the  input  space  and  the  space  W  as  the  output 
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space  be  an  invertible  homomorphic  system.  If  the  original  system  had  more  than  one 
choice  for  the  output  operations,  however,  this  invertible  system  would  also  have  more 
than  one  choice  for  the  output  operations,  which  contradicts  Theorem  22. 

Bee  i  :-e  of  Theorem  27,  the  output  operations  of  a  homomorphic  system  which  are 
consti  ucted  according  to  Eqs.  21  and  22  represent  a  unique  choice  for  these  operations. 
Consider,  for  example,  the  system  having  the  transformation  of  Eq.  26,  with  the  input 
vector  space  being  a  space  under  multiplication.  It  has  been  determined  that  this  system 
is  homomorphic.  An  element  v  o  N  in  the  space  V/N,  where  N  is  the  null  space,  is 
the  set  of  elements  in  V  that  are  scalar  multiples  of  v.  The  mapping  1JT  from  V/N  to 
W,  and  the  inverse  of  4>  are  defined  by 

~  r  i  1  dv 

*lCvJ  =  TdTs  w 

and 

r1W  =  e,,dt=[c»l. 

Using  Eqs.  21  and  22,  we  find  that  the  output  operations  of  the  system  must  be 

~r — i  — i  ~] 

W1  °  w2  =  ^  ^  (w2)  | 

.  r  ,  Jw.dtw  Jw,dtv  1 

■>[('  1  X*  2  + 

and 

j  r /  Jw.dtvC-1 

c/wl  =  ^H(e  )  J  =  cwx 
for  any  and  w2  in  W,  and  any  scalar  c  in  the  field. 
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IV.  CANONIC  REPRESENTATION  FOR  HOMOMORPHIC  SYSTEMS 


4.  1  INTRODUCTION 


The  class  of  homomorphic  systems  has  been  introduced.  The  generalized  principle 
of  superposition  which  is  satisfied  by  these  systems  is  similar  m  form  to  the  principle 
of  superposition  describing  a  linear  system;  when  the  input  and  output  operations  of  a 

homomorphic  system  are  addition,  the 
system  is  linear  and  the  generalized 
principle  of  superposition  reduces  to 
the  statement  of  superposition  in  its 
usual  form. 

A  canonic  representation  for  homo¬ 
morphic  systems  will  now  be  derived. 
This  representation  amounts  to  a  sub¬ 
stitution  of  variables  which  reduces  the 
system  to  a  linear  system.  For  any 
particular  homomorphic  system  the  input  and  output  variables  that  are  chosen  so  that 
the  equivalent  system  will  be  linear,  are  dependent  only  on  the  input  vector  spaces,  and 
not  on  the  details  of  the  homomorphic  mapping  between  these  spaces. 

As  an  example  of  the  form  that  the  canonic  representation  takes,  consider  a  homo¬ 
morphic  system  having  multiplication  as  both  the  input  operation  and  the  output  opera¬ 
tion,  as  indicated  in  Fig.  3.  This  system  has  the  property  that  for  any  two  inputs  v^  and 
v^  and  any  scalar  c. 


Fig.  3.  Homomorphic  system  with  multi¬ 
plication  as  the  input  operation 
and  output  operation. 


[<t» <Vj )]C  <Mv2)  =  wjCw2’ 


(28) 


where  Wj  =  4>(Vj),  and  w2  =  4>(v2). 

If  we  perform  the  substitution  of  variables 

x  =  In  v  (29) 

and 

y  =  In  w,  (30) 

then  the  relation  between  x  and  y  becomes 

=  4>(ex) 
or 


y  =  In  [<t>(ex)  ]  =  4>'  (x) 


(31) 
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But  thC  traiiSfOriuatiOu  4>*  linear  uc^rtUoc  ii  A  |  efIAU  ai  tLliy  tw  U  illJJUlJj  iiHU  U  Its  HUY 


scalar 


<t>'[cxj+x2]  =  Cffc'CXj)  +  4>'(x2). 


Hence  by  the  substitution  of  variables  defined  by  Eqs.  29  and  30,  the  system  has  been 
reduced  to  a  linear  system.  The  system  of  Fig.  3  is  thus  representable  in  the  form  of 
Fig.  4  where  the  linear  transformation  4>'  is  defined  by  Eq.  31.  In  the  representation 


Fig.  4.  Equivalent  representation  of  the  system  of  Fig.  3. 

in  Fig.  4,  the  first  and  last  systems  depend  only  on  the  input  and  output  vector  spaces; 
they  do  not  depend  on  the  details  of  the  transformation  4>.  We  notice  also  for  this  par¬ 
ticular  example,  that  all  of  the  memory  in  the  system  of  Fig.  3  is  concentrated  in  the 
linear  system  <j>'  of  Fig.  4,  since  the  first  and  last  systems  in  this  representation  are 
memoryless.  Here  we  shall  derive  necessary  and  sufficient  conditions  under  which  this 
can  be  done. 

4.2  CANONIC  REPRESENTATION 

In  the  example  just  presented,  a  homomorphic  system  was  reduced  to  a  linear  sys¬ 
tem  by  means  of  a  substitution  of  variables.  This  essentially  requires  a  mapping  of  the 
vector  space  of  inputs  onto  a  vector  space  under  addition,  in  a  one-to-one  manner. 
Similarly,  the  vector  space  of  outputs  must  also  be  mapped  onto  a  vector  space  under 
addition  in  a  one-to-one  manner.  Our  objective  now  is  to  show  that  for  any  vector  space 
such  a  mapping  exists,  and  to  discuss  the  resulting  representation. 

Let  us  restrict  the  system  inputs  to  constitute  a  Hilbert  space  with  an  orthonormal 
basis,  and  restrict  the  system  transformation  to  be  a  continuous  homomorphic  transfor¬ 
mation.  Under  these  restrictions,  the  system  outputs  will  also  constitute  a  Hilbert  space 
with  an  orthonormal  basis.  This  is  equivalent  to  assuming  that  the  input  and  output 
spaces  can  be  approximated  arbitrarily  closely  by  finite  dimensional  spaces.  By  virtue 
of  Theorem  20,  both  the  input  vector  space  and  the  output  vector  space  are  isomorphic 
with  any  infinite  dimensional  Hilbert  space  having  an  orthonormal  basis.  In  particular, 
each  of  these  spaces  will  be  isomorphic  with  a  Hilbert  space  in  which  vector  addition 
is  interpreted  as  the  sum  of  the  corresponding  time  functions,  for  example,  the  space 
of  functions  that  are  square  integrable.  Thus  we  can  always  define  an  invertible 
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homomcjphic  system  that  maps  the  inputs  onto  a  Hilbert  space  under  addition,  and  an 
invertible  homomorphic  system  that  maps  the  outputs  onto  a  Hiloert  space  under  addition. 
Consider  a  homomorphic  system,  then,  with  input  vector  space  V  which  is  a  Kilbert 
space  with  an  orthonormal  basis,  output  vector  space  W,  and  continuous  system  trans¬ 
formation  4>.  Let  denote  the  system  transformation  of  an  invertible  homomorphic 


1 


i  -  -  -  j 

1 _ J 


Fig.  5.  Equivalent  representation  of  a  homomorphic  system. 


system  with  v  as  the  input  vector  space,  and  a  vector  space  under  addition,  denoted  by 
V  ,  as  the  output  vector  space.  Similarly,  let  a  D  denote  the  system  transformation  of 
an  invertible  homomorphic  system  with  W  as  the  input  vector  space,  and  a  vector  space 
under  addition,  denoted  by  W+,  as  the  output  vector  spa  e.  Since  the  systems  aQ  and 


Fig.  6.  Canonic  representation  of  a  homomorphic  system. 

aa  are  invertiole,  the  system  <j>  can  be  represented  in  the  form  shown  in  Fig.  5.  The 
system  enclosed  in  the  dotted  lines,  however,  is  a  linear  system  and  hence  the  system 
of  Fig.  5  can  be  redrawn  in  the  form  of  Fig.  6,  where  L  is  a  linear  system.  We  recog¬ 
nize  furthermore  that  the  system  of  Fig.  6  is  a  homomorphic  system  with  V  as  the  input 
vector  space  and  W  as  the  output  vector  space,  for  any  choice  of  the  linear  system  L. 
The  representation  of  Fig.  6  will  be  referred  to  as  a  canonic  representation  of  homo¬ 
morphic  systems. 

An  example  of  the  canonic  representan.cn  of  a  homomorphic  system  was  shown  in 
Fig.  4.  In  this  case  the  homomorphic  system  had  multiplication  as  both  the  input  and 
output  operations.  An  example  of  such  a  system  is  that  system  having  a  transformation 
<*>  defined  as 

6{v)  =  v  ,  k  ?  real  numi.  r. 

In  this  case,  the  canonic  representation  takes  the  form  of  Fig.  7.  To  obtain  any 
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1 _ J 

Fig.  7.  Canonic  representation  of  a  homomorphic  system  with 
transformation  <J>[v]  =  vk. 

homomorphic  system  having  the  same  input  and  output  vector  spaces  as  the  system  of 
Fig.  7,  we  need  only  replace  the  amplifier  of  gain  k  with  other  linear  systems. 

From  the  form  of  the  canonic  representation,  we  recognize  it  as  a  substitution  of 
variables  which  reduces  a  homomorphic  system  to  a  linear  system.  The  particular 
choice  for  the  substitution  of  "ariables  associated  with  any  specified  homomorphic  sys¬ 
tem  is  governed  only  by  the  input  vector  space  and  output  vector  space  associated  with 
the  system.  The  details  of  the  mapping  between  these  spaces  is  then  contained  in  the 
linear  system  L. 

4.3  CLASSIFYING  HOMOMORPHIC  SYSTEMS 

The  canonic  representation  suggests  a  means  for  classifying  homomorphic  systems. 
Specifically,  let  us  classify  homomorphic  systems  according  to  their  input  and  output 
spaces.  Since  the  characteristic  systems  aQ  and  a  D  are  determined  entirely  by  the  input 
vector  space  and  output  space  associated  with  the  homomorphic  system,  all  systems 
within  a  specified  class  will  have  identical  systems  aQ  and  a  a  in  the  canonic  represen¬ 
tation.  The  systems  within  each  class  will  differ  only  in  the  details  of  the  linear  sys¬ 
tem  appearing  in  this  representation. 

As  a  simple  example  of  this  means  oi  'lassifying  homomorphic  systems,  we  would 
consider  all  linear  systems  as  representing  one  class  of  homomorphic  systems.  Another 
class  of  homomorphic  systems  would  be  the  class  having  the  characteristic  systems  that 
appear  in  the  representation  of  Fig.  7.  In  this  case  vector  additior  is  defined  as  multi¬ 
plication  of  the  associated  time  functions,  and  scalar  multiplication  is  defined  as  the 
associated  time  function  raised  to  the  scalar  power. 

4.4  CONDITIONS  ON  A 'HOMOMORPHIC  SYSTEM  SUCH  THAT  THE 
CHARACTERISTIC  SYSTEMS  ARE  MEMORYLESS 

In  the  class  of  homomorphic  systems  characterized  by  multiplication  as  both  the 
input  operation  and  the  output  operation,  we  observe  that  the  characteristic  systems  aQ 
and  aa  are  memoryless  systems,  i.  e.,  that  output  of  each  at  any  given  instant  of  time 
is  dependent  only  on  the  input  at  the  same  instant  oi  time.  Hence,  for  this  particular 
class  of  systems,  all  of  the  system  memory  can  always  be  concentrated  in  the  linear 
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system  in  the  canonic  representation. 

The  question  arises  as  to  the  conditions  on  a  class  of  homomorphic  systems  under 
which  the  characteristic  systems  aQ  and  aQ  are  memoryless.  Since  these  systems 
represent  invertible  mappings  from  the  input  and  output  vector  spaces  respectively  we 
wish  to  determine  the  necessary  and  sufficient  conditions  on  a  vector  space  consisting 
of  time  functions,  such  that  a  memoryless,  invertible,  homomorphic  mapping  to  a  vec¬ 
tor  space  under  addition  can  be  constructed.  In  the  following  discussion  it  will  be  shown 
that  the  necessary  and  sufficient  conditions  are  that  the  operations  of  vector  addition  and 
scalar  multiplication  must  themselves  be  memoryless. 

To  show  first  of  all  that  these  conditions  on  the  vector  space  are  necessary,  consider 
an  invertible  homomorphic  system  with  transformation  aQ,  input  vector  space  V  and 
output  vector  space  W.  Let  the  operation  of  vector  addition  in  V  be  denoted  by  o  and 
the  opeartion  of  scalar  multiplication  in  V  be  denoted  by  >.  Let  W  be  a  vector  space 
under  addition.  Then  by  virtue  of  Theorem  22,  vector  addition  and  scalar  multiplication 
in  V  must  be  expressible  uniquely  as 

Vi  OV2  =  ao1K{vl)+ao(v2)3  (32) 

SV1  =  tto1tCco(vl)l  (33) 


for  any  v  and  v  in  V  and  any  scalar  c.  Let  us  denote  the  instantaneous  value  of  a 

*  ^  i 

vector  v  in  V  at  a  time  t.  by  vj  .  Then  we  wish  to  determine  wheter  Eqs.  32  and 

1  ll 

33  require  that  the  operations  o  and  >  be  defined  on  the  instantaneous  value  of  v^ 

and  v„  if  a  is  memoryless.  Consider  first,  Eq.  32.  Since  a  is  memoryless,  its  inverse 
£  o  o 

aQ  is  also  memoryless.  Hence,  the  right-hand  side  of  (32)  is  defined  on  the  instan¬ 
taneous  values  of  v^  and  v 2.  Hence,  the  left-hand  side  of  (33)  must  also  be  defined  on 
instantaneous  values  of  v^  and  v^.  We  must  next  ascertain  that 

(vl°v2]lt1=(vl|t1)0(v2i!1)- 

But 

[yi  OV2J<t1={ao1fao(vi)+a0(v2)l}ft1 

ana 

(v1|t.)of2it.)=«;,K(v1i,1)-o(''2i..)} 


(34) 

(35) 


But,  since  aQ  is  memoryless,  Eq.  36  becomes 


(v.l«i)  °  C2l*i) 


Since  addition  is  memoryless,  we  see  that 


i 

\ 
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I  .  ~  I  \  _  /{_  1 \_L^  /..I'll  \ 

(vi|tj)  u  (''zftj  “  “O  (’,VTr"“o'T2,Jit1)' 

Finally,  since  c^1  is  memoryless, 

(vl|t,)  °  (v2  i  t j)  "'■o‘K(v1H',o(v2I1Ii1'  1371 

Comparing  Eqs.  35  and  37,  we  see  that 
tvi  °  v2]!t,  “  (vi!tj)  °  (v2|tl) 

whenever  is  a  memoryless  system.  Similarly,  we  wish  to  show  that  the  assumption 
that  aQ  is  memoryless  implies  that  >  is  rnemoryless,  that  is 


lc>viil«1=[s(vilt1)'- 


Referring  to  Eq.  33,  we  see  that  the  operation  >  is  defined  on  instantaneous  values  of 
Vj,  since  aQ  is  memoryless.  To  show  that  (38)  is  satisfied,  we  observe  that  because 
scalar  multiplication  is  memoryless  and  aQ  is  memoryless. 


But,  the  left-hand  side  of  (39)  is 

"°,Nv>i‘,)]=c>(v>i‘I) 

and  the  right-hand  side  of  (39)  is 

{«o1[c‘o<vi,l},tI  =  (c>(vi)]!t1-  ,40) 

Comparing  Eqs.  39  and  40,  we  see  then  that  Eq.  38  is  satisfied;  consequently,  >  is  a 
memoryless  operation.  Hence,  a  necessary  condition  on  the  vector  space  V  so  that  aQ 
is  a  memoryless  system  is  that  the  operations  o  and  >  must  be  memoryless  operations. 
The  next  step  is  to  show  that  this  also  constitutes  a  sufficient  condition.  To  this  end,  let 
V  be  a  vector  spac  »  with  vector  addition  denoted  o  and  scalar  multiplication  denoted  >. 
Elements  of  the  vector  space  are  time  functions,  that  is,  they  are  scalars  indexed  over 
time  t.  Vector  addition  and  scalar  multiplication  in  V  are  assumed  to  be  memoryless, 
that  is,  at  any  value  of  t. 


|vi0,A=  (vi!t)  °  (^tt) 

(c>v)!t=  c>(vilt) 


for  any  v^  and  v^,  in  V  and  any  scalar  c  in  F.  Let  e  denote  the  identity  in  V. 

Consider  the  set  of  all  scalar  values  which  the  time  functions  in  V  can  assume  at  an 
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instant  of  time  t.  Let  this  set  be  denoted  S..  Then  the  set  S.  is  itself  a  vector  space,  for 

i  i 

(a)  Let  t|}  and  by  any  scalars  in  St*  Then  there  is  a  vector  v^  in  V  and  v 2  in  V 

such  that  Vj  and  v^j  ^  =  h2*  Since  V  is  a  vector  space,  Vj  o  v^  is  in  V; 

hence,  (v}  0V2Ht  is  in  ^t‘  ^ut  ^rom  ^q.  41»  (Vj  v2 ) J t  =  °  r\^<  hence, 

(pj  o  r|„)  is  in  $t.  By  similar  reasoning,  it  can  be  seen  that  ^  o  n2  =  h,  °  ri| 
and  that  (t}}  o  r^)  o  n3  =  hj  o  (^  o  t^)  for  any  t^,  and  ^  in  St. 

(b)  There  is  a  unique  element  et  in  St  such  that  q  o  et  =  tj  for  any  q  in  S^.  To  show 
the  existence  of  such  an  element,  let  v  be  an  element  in  V  such  that  vj^  =  q.  If 
e  is  the  identity  in  V,  then  v  o  e  =  v.  Hence  (v  o  e)|t  =  vjt>  or  (vj^)  o  (e|t)  =  vj^. 
Since  vj  =  q,  we  have 


n  o  (ejt)  =  r). 

Thus  e[t  will  be  an  element  in  such  that  n  o  (ejt)  =  t|,  that  is,  et  =  ejt.  To  show 
that  the  element  et  in  is  unique,  consider  two  elements  et  and  e|  in  with 
the  property  that 

t)  o  et  =  t|  (43) 

and 

t)  o  ej  =  (44) 

for  all  ti  in  St-  Then,  by  virtue  of  (43), 

e^  o  et  =  (45) 

and  by  virtue  of  (44), 

et  o  e|  =  et-  (46) 


But  we  have  stated  that  o  was  a  commutative  operation  in  S^;  hence. 


et 


et* 


Therefore,  from  Eqs.  45  and  46,  et  =  e|. 

(c)  Every  element  n  inS,  has  a  unique  inverse  in  S,.  For,  let  v  be  the  vector  in 

^  —  1  * 

V  such  that  v|  =  rj.  Then,  if  v  denotes  the  inverse  of  v  in  V,  we  have 
v  o  v  =  e;  hence,  (v|t)  o  (v  )j  t  =  e|t>  or  q  o  (v  )j  t  =  et-  Denoting  (v  )jt*>y 

t|~*,  we  have  n  o  q  *  =  e,.  To  show  that  this,  inverse  is  unique,  let  •*!  *  and 

—  1  ^  —I  — } 

T)  denote  two  elements  in  S  such  that  o  ^  =  e.  and  noi).  =  e,.  Then  if 

'  _  j  _j  _  j  t  _  j  1 

t}  =  vjt»  q  =v  Jt  and  =  v  j  t»  it  follows  that 
(v  o  v_1)lt  =(v  o  v”1)  jt. 
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If  Vj 1  is  chosen  by  considering  all  possible  values  for  t.then  v  o  v  *  =  v  o  v}  *  =  e. 

But  the  inverse  of  v  is  unique;  hence  v  1  =  v  1  for  all  values  of  t;  in  particular, 

-It  -It  -1  -1  1 

Vj  |t  =  v  !t  or  t1{  =  n  . 

(d)  The  final  step  in  showing  that  is  a  vector  space  is  to  show  that  scalar  multi¬ 
plication  in  has  the  necessary  properties.  If  rj  is  any  element  in  Sj,  where 

i]  =  vl^,  then  1^  =  l>v)t  =  (1^)1  =  vl^;  hence,  =  n.  By  a  similar  argument 
it  can  be  seen  that 


<°1C2 


and 


(e  1  +c2)>n  =  (c  T1)  o  (c2>ti), 
for  any  Cj  and  c2«  and 

C>^nl  °  ^2*  =  <%V  ° 

for  any  and  n2  in  St  and  any  c. 

If  V  is  a  separable  Hilbert  space,  as  we  have  assumed,  the  vector  space  is  iso¬ 
morphic  with  the  space  Fn,  where  n  is  the  dimension  of  (see  the  Appendix).  Let  this 
isomorphism  be  denoted  c^.  Define  a  vector  space  W  having  elements  that  are  n-tuples 
indexed  over  time,  that  is,  if  w  is  in  W,  then  w|t  is  representable  in  the  form 

(c.,c,,  —  ,c  ),  where  c,,  c_ . c  are  elements  of  the  field.  Then  W  is  a  vector 

space  under  addition,  since  n-tuples  are  added  and  multiplied  by  scalars  according  to 
the  operations  of  addition  and  multiplication  in  the  field.  Now,  there  is  a  mapping  a 
from  V  onto  W  which  is  invertible  and  homomorphic.  Specifically,  let  a  be  defined  by 
the  property  that  [a(v)]jt  =  nt^vIt)  for  any  value  of  t.  Then  a  is  also  a  memoryless 
transformation;  consequently,  it  has  the  properties  that  we  set  out  to  derive.  Hence  we 
have  shown  that  a  sufficient  condition  on  a  vector  space  V  so  that  a  memoryless,  homo¬ 
morphic,  invertible  transformation  from  V  to  a  vector  space  under  addition  exists  is 
that  the  operations  of  vector  addition  and  scalar  multiplication  in  V  be  memoryless. 

To  summarize,  we  see  that  in  the  canonic  representation  of  a  homomorphic  system 

the  system  a  can  be  memoryless  if  and  only  if  the  operations  o  and  >  are  memoryless 

°  -1 
operations.  Similarly,  the  system  a0  (or  equivalently  cQ  )  can  be  memoryless  if  and 

only  if  the  operations  o  and  /  are  memoryless  operations.  In  the  canonic  representation 

for  systems  in  a  class  specified  by  memoryless  input  and  output  operations,  all  of  the 

memory  in  the  systems  can  always  be  concentrated  in  the  linear  portion  of  the  canonic 

representation.  In  contrast,  if  either  the  set  of  input  operations  or  the  set  of  output 

operations  is  not  memoryless,  then  the  memory  of  systems  in  such  classes  can  never 

be  concentrated  only  in  the  linear  portion  of  the  canonic  representation. 

Before  proceeding  to  some  examples  of  the  construction  of  memoryless  invertible 
homomorphic  transformations  from  the  input  and  output  vector  spa-es  of  a  homomorphic 
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system  to  spaces  under  addition,  it  would  be  well  to  investigate  in  more  detail,  the  prop- 
ties  of  these  transformations  and  the  consequences  in  terms  of  the  canonic  representation. 

When  the  canonic  representation  was  first  presented,  the  linear  system  was  inter¬ 
preted  as  a  system  having  a  single  input  and  a  single  output.  We  have  seen,  however,  that 
when  we  desire  the  systems  cq  and  cG  to  be  memoryless  systems,  the  input  and  output 
of  the  linear  system  may  not  be  a  simple  time  function,  that  is,  they  may  not  simply  be 
a  set  of  scalars  indexed  over  time.  Specifically,  consider  a  homomorphic  system  with 
memoryless  input  and  output  operations.  Consider  this  system  in  its  canonic  repre¬ 
sentation  with  the  systems  a  and  aQ  constructed  to  be  memoryless.  Then  the  output  of 
cq  will  be  of  the  form  (f  j{t),  f^ft), . . . ),  where  fj(t),  f^(t)  ...  are  each  scalars  indexed 
over  time.  Similarly,  the  set  of  inputs  to  a  D*  will  be  of  the  form  (gj(t),  g2(t),  *  •  *  )>  where 
gj(t),  g2(t),  . , .  are  each  scalars  indexed  over  time.  In  this  case,  we  would  interpret 
the  system  gq  as  a  memoryless  system  having  a  single  input  and  multiple  outputs.  For 
any  input  v  the  individual  outputs  would  be  the  time  functions  fj(t),  f2(t),  corre~ 
sponding  to  the  mapping  of  the  input  at  any  instant  of  time  onto  an  n-tuple  of  scalars. 
Similarly,  each  output  of  aD^  would  arise  from  the  effect  of  a  set  of  inputs  gj(t)»  g2<t), 

. . .  that  correspond  at  each  instant  of  time  to  the  representation  of  the  output  as  an 
m-tuple  of  scalars.  With  this  interpretation,  the  canonic  representation  of  Fig.  6  would 


Fig.  8.  Canonic  representation  of  a  homomorphic  system  with 
memoryless  input  and  oi  out  operations. 


appear  in  the  form  of  Fig.  8.  In  this  interpretation  the  system  L  has  n  inputs  and  m 

outputs.  It  is  linear  in  the  sense  that,  if  gj,  g2,  ....  gm  are  the  responses  to  a  set  of 

inputs  f.,  f,.  —  f  ,  and  g’  ,  g'  ,  — ,  g'  are  the  responses  to  a  set  of  inputs  f. ,  fl, 

...»  f  ,  then 
n 

(g,+g'i).  <g2+ei> . igm+g^) 

will  be  the  responses  to  the  inputs 

(f.+fj,  (F+fi) . (f  +rj. 

li  z  z  n  n 

and 

cgj,  cg2,  ....  cgn 
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will  be  the  responses  to  cf.,  cL,  ....  ef^  for  any  scalar  c  and  for  any  set  of  inputs. 

If  an  inner  product  can  be  defined  on  the  vector  space  St>  then  the  output  of  cq  at 
time  t  can  be  described  as  the  result  ot  the  inner  product  of  v  with  each  of  the  basis 
vectors  of  S^.  Specifically,  let  the  inner  product  of  v  j }  t  and  v^  j  t  be  denoted  {v^  v2 'V' 
Let  pJt,  . .  .  denote  a  basis  of  S^.  Then  any  scalar  value  v|t  in  can  be  written 

vlt  "  [<vlf  PitMitl 0  t(v|f  0  •  •  •  • 

The  n- tuple  representing  v!t  is,  then,  (Cj,c2, _ _  cn),  where  c  .  s  (y|  ,  p  ).  In  general, 

J  1 

for  different  values  of  t,  the  spaces  S  will  differ;  hence,  both  the  inner  product  and  the 
set  of  basis  vectors  p. ,,  p_, ,  . . .  will  be  functions  cf  time  Let  us  denote  the  operation 
of  taking  the  inner  product  of  the  input  at  any  instant  of  time  t  with  the  j  n  orthonormal 
basis  vector  in  St  by  j t(v | ^),  that  is, 

Then,  as  we  consider  all  instants  of  time,  we  have  constructed  a  function  H.(v)  with  the 
property  that 


[Hj(v)]!t  =  Hjlt(v|t>, 


By  referring  to  the  definition  of  the  outputs  of  cr,  we  see,  then,  that 
H.(v)  =  f  (t>  j  =  1,  2,  ....  n. 

J  J 

To  construct  the  nverse  of  H^,  we  observe  that  the  inverse  of  H^|t  corresponds  simply 
to  the  combination  of  H ^  J  ^ { v  |  ^ )  with  the  basis  vector  p^  according  to  the  operation  of 
scalar  multiplication  in  the  vector  space  V,  that  is, 


Hjir’w-sv 


Hence,  the  inverse  of  is 


H~ 1  [f(t>]  -  f(t)>p.(t>. 


where  p.{t)  denotes  the  j  ‘  orthonc-rmal  basis  vector  in  S,  as  a  function  of  t.  Interpreting 

1  _  j 

Fig.  8  in  the  light  of  this  discussion,  then,  we  can  replace  the  systems  oq  and  aD  as 
shown  in  Fig.  9. 

As  an  example  of  the  application  of  these  ideas  to  the  construction  of  the  systems  a 

-1  ° 
and  aD  ,  consider  a  homomorphic  system  with  addition  as  the  input  operation  and  addition 

as*  the  output  operation,  that  is,  a  linear  system.  The  operations  of  vector  addition  and 

scalar  multiplication  on  the  input  vector  space  are 
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and 


c^v  =  cv. 

Let  denote  the  set  of  all  possible  values  that  the  input  can  assume  at  time  t.  We  have 
shown  that  St  is  a  vector  space.  In  particular,  it  is  a  one-dimensional  vector  space. 


Fig.  9.  Canonic  representation  of  a  homomorphic  system  with  memoryless  input 
and  output  operations,  illustrating  the  construction  of  the  characteristic 
systems. 

For,  choose  any  nonzero  value  (3  in  St  as  a  basis  vector,  then  any  value  rj  in  can  be 
expressed  in  the  form  rj  n  c^p  =  cp  for  some  scalar  c  in  the  field.  Specifically, 

c  =  ^(n), 

where  4-  is  well  defined,  since  it  is  a  nonzero  scalar  in  the  field,  and  hence  has  an 
P 

inverse  under  multiplication.  Thus  the  vector  p  in  spans  S^..  Let  an  inner  product 
be  defined  in  St  as  follows:  if  and  are  any  two  vectors  in  St»  then  the  inner  prod¬ 
uct  of  •Hj  and  ^2>  denoted  by  ('Hj*  Tl2}  will  be  defined  as 

(VVS  V2* 

If  the  field  is  the  field  of  real  numbers  rather  than  the  field  of  complex  numbers,  then 

{VV  =  ’’l’V 
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To  verify  that  Eq.  47  defines  an  inner  product,  we  refer  to  Definition  5.  Let  Pj, 
and  be  any  vectors  in  S^.  Then 

(1)  ((D,  +t>2)»  ^3)  =  (tIj  +t12)  h3 
and  thus 

((nj+^J.  ^13)  =  (hj.hj)  + 

(2)  (ct)  ,  ti2)  =  crijT^  =  c  •  (rjj.  p2). 

(3)  (*ij.  ^2)  =  (-n2.  nj) 

(4)  (T»1*T»j)  =  rij'n'i  >  0,  r|j  9t  0. 

An  orthonormal  basis  for  S  under  this  choice  for  the  inner  product  is  the  choice  j?  =  1. 

t  I 

We  are  guaranteed  tha*  the  scalar  1  is  in  St;  for,  if  n  is  any  scalar  in  then  —  must 

be  a  scalar  in  the  field,  and  since  S  is  a  vector  space,  the  scalar 


must  be  in  St« 

Since  is  a  one-dimensional  vector  space,  the  system  aQ  has  a  single  output.  The 
operation  Hjt(vjt)  is  given  oy 

Hft(v|t)  =  (v|t,  p)  =  v|t, 

and  hence  H(v}  =  v.  Consequently,  the  system  cq  is  simply  the  identity  system,  as  would 
have  been  expected  without  recourse  to  this  formalism.  By  a  similar  procedure,  the 
system  aQ  and  hence  aD  is  the  identity  transformation. 

The  choice  of  inner  product  specified  by  Eq.  47  is  by  no  means  a  unique  choice.  As 
the  inner  product  changes,  the  orthonormal  basis  also  changes.  In  terms  of  a  different 
inner  product,  the  systems  oq  and  aD  will  be  different.  Consider,  for  example,  the 
inner  product  defined  as 

(vVs  kT]i\ 

for  some  positive  real  scalar  k  in  the  field.  It  can  be  verified  that  this  inner  product 
has  the  properties  required  of  an  inner  product.  Under  this  inner  product,  an  orthonormal 
basis  for  St  is  the  vector 

p =  k-‘/2. 
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The  operator  H|t(v|t)  is  then  given  by 

H|t(vjt)  =  (v|t.p)  =  kl/2  v|t 
and  hence 

H(v)  =  k1/2  v. 

l/2 

Thus  the  system  a  is  an  amplifier  of  gain  k  '  .  If  this  same  inner  product  is  associated 

0  -1  -l/2 

with  the  output  vector  space,  then  the  system  a  Q  will  be  an  amplifier  of  gain  k  '  . 

Alternatively,  different  inner  products  can  be  associated  with  the  inpu;  and  output  vector 

spaces;  the  systems  aQ,  L  and  aj  will  depend  in  general  on  the  manner  in  which  these 

inner  products  are  defined. 

As  another  example.,  consider  the  class  of  systems  having  an  input  vector  space  V 
with  vector  addition  and  scalar  multiplication  defined  as 


V2  = 


V  V 
12 


SV1 


for  all  Vj  and  v^  in  V  and  all  scalars  c  in  the  field.  Let  the  field  be  restricted  to  the 
field  of  real  numbers,  and  the  time  functions  be  restricted  to  have  only  positive  values. 
Again,  will  denote  the  set  of  values  which  the  inputs  can  assume  at  time  t.  St  is  a 
one-dimensional  vector  space.  For,  let  p  be  any  vector  in  St  other  than  unity,  then  for 
any  vector  ri  in  St  there  exists  a  scalar  c  in  F  such  that 

SP  =  [Pf  =  ^  *48} 

Tliis  follows  from  the  fact  that  all  scalars  in  must  be  positive.  Hence  the  natural 
logarithms  of  p  and  q  are  defined,  and  the  natural  logarithm  of  p  is  nonzero,  since 
P  is  not  equal  to  unity.  The  choice  of  the  scalar  c  in  Eq.  48  is  given  by 

ln(q) 

In  (p) " 

In  the  previous  example,  the  system  aQ  was  constructed  by  first  defining  an  inner  prod¬ 
uct.  Let  us  instead,  in  this  example,  determine  a  by  first  constructing  the  inverse 
aQ*.  We  know  that  this  approach  can  be  used,  since  there  is  always  some  inner  prod¬ 
uct  under  which  the  basis  for  is  orthonormal  when  the  dimension  of  S t  is  finite,  as  it 
is  in  this  example.  Hence  we  do  not  need  to  define  an  inner  product  in  St  in  order  to 

determine  an  orthonormal  basis  for  S..  From  the  previous  discussion  we  see  that  the 

-1  1 
system  oQ  is  described  by 
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iiiW  MIMohIbhm 


a  *{x)  =  X.  0 

O''  >' 

where  x  =  cQ(v)  for  some  input  v.  Since 

*>f<  =  [Pf. 

we  have 

a“!(x)  =  [pf. 

We  recognize  the  inverse  of  this  system  as  the  logarithm  to  the  base  p  and  hence 
aQ[v]  =  logp  [vj. 

It  would  be  instructive  to  determine  the  inner  product  under  which  p  is  orthonormal.  To 
do  this,  consider  the  fact  that  with  the  above-mentioned  choice  of  co>  the  operator  H|^ 
is  given  by 

H I  t(v  1 1)  =  l°gp  [v|t]. 

Hence,  from  the  definition  of  H|t> 

(v|t,  p)  =  logp  [v|t]. 

Consider  the  inner  product  of  any  two  vectors  Tij  and  in  st  given  by 

(vV =  [logp(T,l)][logp(712)J-  <49> 

Before  showing  that  p  is  orthonormal  under  this  inner  product,  let  us  first  verify  that 
(49)  does  in  fact  define  an  inner  product. 

(1)  =  [logp(TliV][logp(V] 

=  [logpTl1+logpT12][logp(tl3)] 

=  (logp’ljHlogpnj)  +  U°gpTi2){iogpT1.i) 

and  thus 

h3)  =  +  <T|2,  Tl3>. 

(2)  tj2)  =  c(logpiij)(logpTj2)  =  cfiij,  t^) 

(3)  Property  (3)  does  not  have  to  be  verified,  since  the  field  has  been  restricted  to 
the  field  of  real  numbers. 
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(4)  (nj.^j)  =  (logpi^)''  >  0  for  Tjj  *  1  as  required. 

It  follov/s  directly  that  the  vector  fJ  is  oriho  ormal  under  this  inner  product  for 
(p,  p)  =  [logp(P)]2  =  1. 

4.5  APPLICATION  TO  NONADDITIVE  FEEDBACK  SYSTEMS 

The  notion  of  homomorphic  systems  can  be  used  to  transform  some  systems  with 
nonadditive  feedback  to  systems  with  additive  feedback.  In  certain  special  cases  this 
transformation  has  the  effect  of  removing  the  system  nonlinearities.  In  such  cases, 
stability  of  the  nonlinear  feedback  system  can  be  investigated  by  using  the  techniques 
available  for  the  study  of  linear  feedback  systems. 


Fig.  10.  Homomorphic  feedback  system  with 
nonadditive  feedback. 


Consider  a  feedback  system  of  the  form  shown  in  Fig.  10.  The  inputs  are  assumed 
to  be  a  Hilbert  space  with  an  orthonormal  basis,  in  which  vector  addition  is  the  oper¬ 
ation  o.  The  system  4>  is  a  homomorphic  system  with  input  operation  o  and  output 
operation  o  .  The  system  y  is  a  homomorphic  system  with  input  operation  o  and  out¬ 
put  operation  o.  The  feedback  operation  is  identical  to  the  input  operation  of  4>  and  the 
output  operation  of  y. 

The  first  step  in  the  transformation  of  the  sys,rm  e*  Fig.  10  is  to  replace  <j>  and  y 
with  their  canonic  representations.  We  recognize  that  since  the  input  operation  of  4>  is 
identical  to  the  output  operation  of  y,  the  first  system  in  the  canonic  representation  for 
<J>  can  be  chosen  as  the  last  system  in  the  canonic  representation  for  y.  Similarly,  since 
the  output  operation  of  $  is  identical  to  the  input  operation  of  y,  the  last  system  in  the 
canonic  representation  for  4>  is  identical  to  the  first  system  in  the  canonic  representa¬ 
tion  for  y.  When  $  and  y  are  replaced  by  their  canonic  representations,  the  system  of 
Fig.  11  results.  From  Fig.  11  we  see  that  the  input  to  the  linear  system  Lj  is 
given  by 
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or,  since  c  is  s  homomorphic  system, 

X2  =  a0(v)  +  y2* 

The  input  y^  to  the  linear  system  L 2  is  given  by 

Hence,  the  block  diagram  of  Fig.  11  can  be  transformed  to  that  shown  in  Fig.  12.  From 

the  block  diagram  of  Fig.  12  we  recognize  the  feedback  system  as  a  cascade  of  a  »  a 

-1  ° 
linear  system,  and  aQ  .  Hence,  it  is  a  homomorphic  system  with  o  and  o  as  the  input 


Fig.  11.  Equivalent  representation  of  a  homomorphic  feedback  system. 


and  output  operations,  respectively.  The  canonic  representation  for  this  homomorphic 
system  is,  of  course,  the  block  diagram  of  Fig.  12.  The  linear  portion  of  the  canonic 
representation  is  a  linear  feedback  system  having  the  linear  portion  of  «j>  in  the  forward 
path  and  the  linear  portion  of  y  in  the  feedback  path. 

The  systems  aQ  and  in  the  system  of  Fig.  12  are  determined  by  the  classes  to 


Fig.  12.  Canonic  representation  of  a  homomorphic  feedback  system. 

which  the  systems  $  and  y  belong.  Hence,  as  $  and  y  are  varied  within  their  respec¬ 
tive  classes,  the  systems  cQ  and  a  remain  the  same;  changes  in  <$>  and  y  are  repre¬ 
sented  by  changes  in  the  linear  systems  Lj  and  Thus  the  behavior  of  the  feedback 
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system  of  Fig.  10  as  the  systems  <J>  and  y  are  varied  within  their  respective  classes  can 
be  studied  by  concentrating  on  the  behavior  of  the  linear  feedback  system  enclosed  in  the 
dotted  line  in  Fig.  12. 

The  fact  that  the  nonlinearities  in  the  system  of  Fig.  10  can  be  removed  from  the 
feedback  loop  depends  on  the  fact  that  the  feedback  operation  is  identical  to  the  input 
operation  of  <}>  and  the  output  operation  of  y,  and  that  the  output  operation  of  <i>  and  the 

input  operation  of  y  ar<“  identic?1.  Under 
these  restrictions,  and  the  restriction 
that  the  system  inputs  constitutes  a  vec¬ 
tor  space  under  o,  the  system  outputs 
will  constitute  a  vector  space  under  o . 

It  can  also  be  verified  that  the  set  of 
inputs  to  <j>  is  a  vector  space  under  o 
and  the  set  of  outputs  of  y  is  a  vector 
space  under  o. 

If  we  do  not  impose  the  restriction 
that  the  feedback  operation  be  identical 
to  the  input  operation  of  <}>  and  the  output 
operation  of  y,  we  can  still,  in  certain  cases,  transform  the  system  to  a  feedback  sys¬ 
tem  with  additive  feedback.  The  over-all  system  will  no  longer  be  a  homomorphic  sys¬ 
tem;  consequently,  the  feedback  loop  will  remain  nonlinear. 

Specifically,  consider  the  feedback  system  in  Fig.  13.  The  system  inputs  constitute 
part  of  a  Hilbert  space  under  the  operation  O  .  It  is  assumed  that  the  system  of  Fig.  13 
is  well  defined,  that  is,  the  operation  o  is  defined  on  all  inputs  to  4>  and  the  operation 
O  is  defined  on  all  the  outputs  of  y.  This  would  be  true,  for  example,  if  <j>  and  y  were 
both  linear  systems  and  the  operation  were  multiplication. 


Fig.  13.  Nonadditive  feedback  system  with 
homomorphic  systems  in  the  for¬ 
ward  and  feedback  paths. 


Fig.  14.  Equivalent  representation  of  the  feedback  system  of  Fig.  13. 


Since  the  system  inputs  constitute  a  vector  space  under  the  operation  O  ,  there  exists 
an  invertible  homomorphic  system  having  O  as  the  input  operation  a..d  addition  as 
the  output  operation. 

The  input  €  to  4>  is  given  h\ 
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£  =  V  O  X 


or,  equivalently, 

€  =  “olad(v  °  x,l* 

Since  a q  is  a  homomorphic  system,  (50)  can  be  rewritten 

€  =  aoK(v)+ao(x)]' 

Consequently,  the  sys.  ■ .  m  of  Fig.  13  can  be  redrawn  as  shown  in  Fig.  14.  When  the 
canonic  representations  for  <>  and  y  are  substituted  in  the  block  diagram  of  Fig.  14, 
and  appropriate  block  diagram  manipulations  are  performed,  the  system  of  Fig.  14  can 


i - 1 


I _ J 


Fig.  15.  Equivalent  representation  of  the  feedback  system  of  Fig.  13. 


be  transformed  to  that  shown  in  Fig.  15.  If  the  system  y  is  an  invertible  system,  then 
the  system  will  also  be  invertible.  In  this  case,  the  system  for  Fig.  15  can  be 
transformed  to  that  shown  in  Fig.  1 6,  for  which 


L=Ljl2‘ 


p  = 


a 


-1 
a  . 
O  o 


If  this  feedback  system  is  to  be  studied  as  the  system  $  is  varied  within  its  class,  then, 
the  systems  a  and  p  will  remain  invariant  with  changes  in  Hence,  properties  of 
the  feedback  system  of  Fig.  13,  as  $  is  varied  within  any  one  class,  can  be  studied  by 
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I - 1 


Fig.  16.  Equivalent  representation  of  the  feedback  system  of  Fig.  13 
for  the  case  in  which  \  is  invertible. 


concentrating  on  the  feedback  system  enclosed  in  the  dotted  line  in  Fig.  16.  Although 
this  feedback  system  is  nonlinear,  it  is  characterized  by  the  property  that  it  is  a  unity 
feedback  system  in  which  the  forward  path  contains  a  linear  system  inserted  between 
a  nonlinear  system  and  its  inverse. 
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APPENDIX 


IcAV»'iA»*rtV)i/t  Mnnn.wM  C  «  O - - U  I  _  »▼••»•  >  n  -  - «  — .  H 

— ~~~  »*qkhaii5  ^  «*  iiuitr  n  uoen  apace  mio  t tie  Space  F~ 


In  Section  IV,  we  were  concerned  with  the  construction  of  an  isomorphic  mapping 
from  the  input  and  output  spaces  of  a  homomorphic  system  into  spaces  under  addition. 
The  input  and  output  spaces  were  restricted  to  be  separable  Hilbert  spaces,  i.e.,  Hilbert 
spaces  having  orthonormal  bases.  When  the  operations  of  vector  addition  and  scalar 
multiplication  were  memoryless  operations,  the  set  of  values  which  the  associated  time 
functions  could  assume  at  any  instant  of  time  was  shown  to  constitute  a  vector  space.  It 
was  then  stated  that  an  isomorphism  could  be  defined  which  mapped  the  set  of  values  at 
any  instant  of  time  onto  the  space  Fn.  The  purpose  of  this  appendix  is  to  carry  out  the 
derivation  of  these  results. 

Consider  a  separable  Hilbert  space  in  which  the  orthonormal  basis  is  denoted  f  (t) , 
f2(t),  ....  Then  if  f(t)  is  any  function  in  the  space,  f(t)  can  be  expressed  in  the  form 


f(t)  =  y"[<fm.fk<t»>fk<t)],  (a.  i) 

o 

where  T  denotes  the  combination  of  the  functions  [(f(t),  yt^yt)]  according  to  the  oper 

ation  o.  Let  S  denote  the  set  of  all  values  that  can  be  assumed  by  the  functions  in  the 

space  at  any  given  time  y  If  o  and  >  are  memoryless  operations,  then,  from 
Eq.  A.  1, 


f<y-  T  [(«»).  fk(t»>fk(t0)]. 

o  1 

Consequently,  for  any  scalar  f(y  in  S  there  exists  a  set  of  scalars  X, ,  X  ,  ...  such 
that  1  2 

f<y  -i\>  w- 

o 1 

The  scalar  X^  is  given  by 
Xk= 

The  vector  space  S  is  spanned  by  the  scalars  f j (tQ) ,  f2<to),  ...  in  the  sense  that  any 
scalar  in  S  is  the  limit  of  a  linear  combination  of  the  scalars  f j (tQ) ,  f  (t  ),  ...  under 
the  operations  o  and  >.  The  set  of  scalars  yy,  yy ,  ...  is  a  countable  set,  since 
an  orthonormal  basis  for  a  separable  Hilbert  space  is  countable.  This  set  of  scalars 
may  or  may  not  constitute  a  basis  for  S,  that  is,  they  may  or  may  not  be  independent. 
They  contain,  however,  an  independent  set  in  terms  of  which  any  scalar  in  S  can  be 
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expressed  as  a  (possibly  infinite)  linear  combination,  as  the  following  argument  shows. 

If  the  set  S  contains  only  one  scalar,  then  this  scalar  must  be  the  identity  in  S 
because  S  is  a  vector  space.  Since  fj(t  ),  f 2 (tQ) »  •  •  ♦  are  each  in  S,  they  must  all  be 
equal  to  the  identity  in  S.  In  this  case,  any  one  of  these  scalars  is  a  basis  for  S,  and 
hence  the  proof  is  complete. 

If  the  set  S  contains  more  than  one  scalar,  then  the  set  f ^  (tQ)  *  f 2 (t0> ,  . . .  must  con¬ 
tain  at  least  one  scalar  that  is  not  the  identity  in  S.  Let  Sj,  s2>  . . .  denote  the  sequence 
of  all  scalars  from  the  sequence  f ^  (t0>  '2(tQ),  . . .  which  are  other  than  the  identity  in  S. 
This  sequence  is  nonempty  by  virtue  of  the  comments  just  made.  Let  Sj  be  the  subspace 
of  scalars  s  in  S  which  is  spanned  by  Sj.  Let  s^  be  the  first  scalar  in  the  sequence 
Sj,  s2,  . . .  which  is  not  in  S^,  and  let  S2  denote  the  subspace  of  S  spanned  by  s*  and 

s^,  where  s'x  s  s^.  Continuing  this  process,  we  obtain  a  sequence  of  subspaces  Sj, 

n  . 

S2,  —  of  S.  Since  S  is  the  limit  of  the  sequence  of  partial  sums  Tj  [(f(t),fj{(t))>fjt(t0)J, 

the  limit  of  the  sequence  of  subspaces  Sj,  S2,  ...  is  S.  Hence  the  sequence  si,  s^,  . . . 
spans  S,  since  the  space  can  be  expressed  as  a  combination  of  s'j,  s^,  . . . ,  under 
the  operations  o  and  >.  Furthermore,  by  the  manner  in  which  the  sequence  s' ,  s!,,  . . . 
was  generated,  it  is  independent.  This  sequence  >o“ms  a  basis  for  the  space  S  in  the 
sense  that  it  is  independent  and  that  any  element  in  S  can  be  expressed  as  the  limit  of 
partial  sums  of  scalars  in  the  sequence. 

We  wish  to  show  next  that  S  is  isomorphic  with  Fn  for  some  n.  Let  s  be  any  scalar 
in  S.  Since  the  sequence  s'j,  s^,  . ..  spans  S  and  since  this  sequence  is  independent, 
s  can  be  expressed  in  one  and  only  one  way  in  the  form 

n 

SII,ck>sk 

o 

for  a  set  of  scalars  {c^}  from  F,  where  c^  =  [(f(t)>f^(t))].  The  isomorphism  T  from 
S  to  Fn  is  then  defined  as 

T(s)  =  (cx,c2,...). 
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